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PREFACE 


Hus  Note  presoits  one  of  seven  case  studies  developed  as  part  of  the  Project  AIR 
FORCE  study  *Managing  Risks  in  Weapon  Systems  Development  Prqjects  ”  vdiich  has 
developed  case  studies  on  the  level,  distrilwtion,  and  risk  in  the  range  of  major  Air  Force 
development  programs  conducted  primarily  during  the  19808.  Notes  based  on  those  case 
studies  offer  concise  descriptions  and  analjrses  of  the  policies  used  by  the  Air  Force  to 
manage  and  distribute  risk.  They  are  aimed  primarily  at  high-level  government  officials 
concerned  with  the  management  of  researdi  and  development,  induding  senior  Air  Force 
staff,  senior  Department  of  Defense  (DoD)  offidals,  and  congressional  staff.  The  Notes 
should  also  be  useful  to  policy  analysts  concerned  with  the  management  of  large-scale 
research  and  development,  espedally  in  DoD. 

This  Note  examines  risks  associated  with  the  program  used  to  develop  a  derivative 
filter,  the  F-16C^.  That  fighter  uses  as  subsystems  many  of  the  other  systems  studied  in 
this  project  This  Note  examines  the  risks  assodated  with  integrating  those  systems  into  the 
F-1€(VD.  The  derivative  development  described  here  has  important  implications  for  future 
polides  that  could  place  greater  emphasis  on  sudi  development  to  fadlitate  quidc  and 
flexible  responses  to  unexpected  changes  in  a  diffuse  threat  Information  is  current  as  of 
summer  1991. 

Other  Notes  written  in  this  project  include  the  following: 

•  S.  J.  Bodilly,  Cose  Study  ofBitk  Management  in  the  USAFB-IB  Bomber 
Program,  RAND,  N-3616-AF,  1993. 

*  S.  J.  Bodilly,  Cate  Study  of  Risk  Management  in  the  USAF  LANTIRN  Program, 
RAND,  N-S617-AF,  1993. 

•  F.  Camm,  The  Development  <^the  FlOO-PW-220  and  FllO-GE-100  Engines:  A 
Case  Study  of  Risk  Assessment  and  Risk  Management,  RAND,  N-3618-AF,  1993. 

*  K.K  Mayer,  The  Development  the  Advanced  Medium  Range  Air-to-Air  Missile: 
A  Case  Study  of  Risk  and  Reward  in  Weapon  System  Acquisition,  RAND,  N-3620- 
AF,  forthcoming. 

Two  related  unpublished  papers  have  been  written  by  T.  J.  Webb  on  risk  management 
during  the  development  of  the  Global  Positioning  System  Blodi  I  satellite  and  risk 
manafement  in  preparing  for  devel«qnnent  of  the  Joint  Surveillance  Target  Attadc  Radar 
Qystem  (Joint  STARS).  A  summary  of  these  Notes  and  papers  and  the  poli^  conclusions 
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based  on  them  is  found  in  T.  K  Glennan  et  al..  Barriers  to  Managing  Risk  in  Large-Scale 
Weapon  System  DeveUgtment  Programs,  RAND.  MR-248-AF,  forthcoming. 

The  Air  Force  sponsor  for  the  study  is  the  Deputy  Assistant  Secretary  of  the  Air  Force 
(Contracting)  (SAF/AQC).  Ihe  work  has  been  conducted  in  the  Resource  Management 
Program  of  Prefect  AIR  FORCE.  The  principal  investigator  at  RAND  is  Dr.  Thomas  K 
Glennan. 


The  F-16  Multinational  Staged  Improvement  Program  (MSIP)  is  the  developmoit 
program  that  the  F-16  program  has  used  to  move  beyfmd  the  F-16A/B.  Its  primary  inoduct 
has  been  the  F-16G/D,  an  aircraft  whose  design  evolves  over  time  as  new  technological 
rypahilitiaa  becomo  available  or  attractive  to  incorporate  in  its  deeigrr.  MSIP  is  the  program 
that  F-16C/D  developers  have  used  to  introduce  these  capabilities  over  time.  The  prime 
contractor  for  the  F-16,  General  Dynamics,  and  the  F-16  System  Program  OfBce  (SPO) 
formally  initiated  the  program  in  1980.  It  continues  today. 

MSIP  provides  a  derivative  approach  to  development,  an  approach  the  Air  Force  may 
want  to  more  broadly  in  the  future  for  several  reasorrs.  The  envirorunent  in  which  the 
Air  F<noe  does  business  is  rfiawging  rapidly.  The  external  threat  that  it  must  engage  is  far 
lai^g  well  tMhunl  than  it  had  been  in  the  past  and  is  likely  to  change  fairly  rapidly  over  time. 
The  Air  Force  will  not  have  the  resources  it  has  today  to  face  such  an  uncertain  threat.  A 
development  process  that  allows  fairiy  rapid  response  to  changes  in  threat  at  a  moderate  cost 
would  appear  to  be  very  attractive.  MSIP  offers  such  a  process. 

This  study  examines  MSIP,  giving  special  attention  to  means  of  assessing  and 
managing  the  risks  associated  with  system  development.  It  is  one  of  seven  case  studies 
conducted  by  RAND  for  the  Air  Force  to  examine  the  Air  Force’s  management  of  risk  in 
development  programs  during  the  19808.  Other  case  studies  address  the  Alternate  Filter 
Engine,  AMRAAM,  B-IB,  Global  Positioning  S3r8tem  (GPS),  JSTARS,  and  LANTIRN 
systems.  MSIP  has  integrated  a  number  of  these  subqrstems  into  the  F-16C/D. 

When  we  speak  of  risk,  we  mean  a  sitoation  in  which  a  manager  can  be  surprised  in  a 
negative  way.  The  tu^ier  the  probalality  of  a  negative  outcome  fiom  an  activity  is,  the 
riskier  that  activity  is.  Development  managers  clearly  try  to  limit  the  probability  of  negative 
outcomes  fixnn  their  highly  uncertain  developmoit  efSnrts.  Development  programs  effectively 
set  a  number  of  minimum  acceptaUe  outcnnes  relating  to  system  performance  and  the  cost 
and  adiedule  of  the  devtiopment  Over  the  course  of  a  development,  its  managers  attempt  to 
«iHmin«fa»  the  possibility  tiiat  the  minimum  outcomes  will  not  be  met.  In  this  sense,  tiieir 
managament  of  risk  is  difficult  to  distinguish  frnn  their  management  ot  development  in 
gmeral.  Development  managers  do  not  make  a  strong  distinction.  Hence,  this  study  views 
risk  sesBaament  and  risk  management  in  rather  broad  terms. 

IffilP  is  called  ‘Multinational”  because  it  induiMs  the  Eunqwan  Participating 
Govamments  (EPG)— Bdghun,  Denmark,  The  Netherlands,  and  Norway— 4n  planning  and 
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dedaioiunaking.  In  this  process,  the  United  States  is  the  dominant  partner.  For  the  most 
part,  MSIP  has  developed  new  versions  of  the  F-16C/D  for  the  U.S.  Air  Force  and  then 
offered  variations  to  the  EPG  and  to  other  foreign  air  forces  not  represented  in  the  process. 
This  approach  limits  both  risks  associated  with  setting  new  system  specifications  risks 
associated  with  imidemoiting  derngns  for  foreign  air  forces.  It  also  affirms  the  U.S.  Air 
Force’s  strong  continuing  interest  in  the  F-16C/D,  assuring  foreign  governments  that  the 
U.S.  Air  Force  will  continue  to  support  tiie  aircraft  designs  they  buy. 

MSIP  is  called  '‘staged”  because  it  incorporates  new  capabilities  in  increments.  It  is  an 
approach  that  allows  MSIP  to  incorporate  new  capabilities  in  the  F-16C/D  as  technology  or 
requirements  change;  to  use  a  retrofit  prt^pram  to  control  production  costs  (MSIP 
inoorp(»rates  provisiona  in  aircraft  produced  in  an  early  stage  that  reduce  the  cost  of 
retrofitting  subsystems  that  will  be  incorporated  in  aircraft  produced  in  a  later  stage);  and  to 
resolve  the  risks  associated  with  the  subsystems  int^rated  during  one  stage  before  moving 
on  to  engage  the  risks  associated  with  subsystems  to  be  integrated  in  the  next  stage.  The 
program  was  initially  conceived  with  three  stages.  General  Pynamics  proposed  a  fourth 
stage  to  implement  a  follow-on  to  the  F-16(}/D;  it  was  rejected  in  1989. 

MSIP  is  essentially  a  management  device  for  coordinating  many  concurrent  efforts  to 
integrate  subsystems  with  one  another  and  an  F-16  airfirame.  That  is,  in  each  stage,  new 
designs  of  the  F-16C/D  are  conceived  that  integrate  many  new  subsystems  to  create  a 
coherent  aircraft  with  new  combat  capabilities.  To  do  so,  MSIP  relies  on  the  F-16  Falcon 
Century  prt^ram  to  survey  new  c^wirilities  and  consider  matches  of  tedmological 
capabilities  and  missions  that  might  be  used  to  define  new  aircraft  designs.  When  it 
discovers  a  new  subsirstem  program  that  xx>i|^t  be  attractive  to  integrate  in  the  future,  MSIP 
establiahes  a  relationship  with  the  program  as  early  as  possible.  MSIP  works  with  the 
program  to  provide  test  assets,  influence  design  specifications  that  affect  the  subsystem’s 
compatibility  with  the  F-16,  and  ultimate^  coordinate  the  integration  of  the  subs3rstem  with 
the  other  subsystems  that  it  will  join  in  a  new  F-16G/D  design. 

To  this  integratioii  task,  MSIP  brings  a  teat-analyw-fiz  approach,  which  emphasizes 
the  need  for  extensive,  iterative  testing  to  yield  quid^y  empirical  infinrmatimi  on  problems. 

As  the  procees  reveals  integration  problems,  MSIP  develtqiers  can  analyze  and  fix  them  and 
then  test  again  for  success  of  fix.  When  it  attonpts  to  integrate  many  subsystems  at  once, 
this  approach  is  demanding  because  subsystem  integrations  proceed  at  differmt  rates.  It  is 
difficult  to  maintain  an  up-to-date  and  coherent  ccmfiguration  against  which  to  test 
simultanooualyeadi  of  the  subsystems  being  integrated.  To  nipport  the  test  effort,  MSIP 
has  enqpkyed  a  Qystsaas  Integration  Laboratory  and  F-16C/D  simulator  at  General 
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Pynamics  to  great  effect  But  the  process  is  still  diallenging.  MSIP  managers  expected  that 
the  greatest  risk  in  their  program  would  arise  from  problems  associated  with  integrating 
many  subsystems  at  once,  and  they  were  rifl^t. 

MSIP,  then,  presents  a  program  in  which  planners  can  anticipate  that  many  changes 
win  occur  in  the  future.  Some  changes  will  occur  simply  because  the  program  is  developing 
new  aircraft  designs  for  implementation.  Others  will  occur  because  the  planned  integrations 
do  not  proceed  as  expected.  To  prepare  for  such  changes,  MSIP  has  deme  two  important 
things.  First,  it  has  established  a  flexible  management  strategy  and  contractual 
environment  that  plan  for  change  to  occur  and  respond  to  individual  changes  as  they  occur. 
The  strategy  and  contractual  environment  affect  the  F>16  SPO,  General  Dynamics,  all  the 
SPOs  and  prime  contractors  associated  with  subsystems  being  integrated  through  MSIP,  and 
the  many  teat  facilities  that  support  MSIP  efforts.  They  focus  as  much  on  establishing  and 
niainfatinin£  good  relationships  among  these  organizations  as  they  do  on  controlling 
individual  changes.  Second,  MSIP  has  assembled  an  experienced  management  staff  to 
handle  dianges  as  they  occur.  Staff  quality  is  more  important  to  MSIP  than  it  is  to  more 
traditional  developments  precisely  because  its  flexibility  caimot  be  effective  imless  MSIP 
managers  respond  effectively  as  MSIFs  development  tasks  diange  over  time. 

Wlule  MSIP  has  overseen  the  F-16C/D  program’s  development  activities,  a  series  of 
three  multiyear  production  contracts  have  governed  its  production  activities.  Those 
contracts  signal  a  strong  consensus  between  the  Congress  and  Air  Force,  both  approvers  of 
the  amtracts,  that  the  F-16  program  would  remain  healthy  and  continue  in  a  fairly 
predictable  way.  Such  consensus  must  have  relieved  MSIP  managers  about  one  migor  risk 
that  developers  must  typically  address — the  risk  that  their  programs  will  not  continue. 

MSIP  managers  could  presumably  give  greater  attention  to  other  risks  associated  with 
developing  the  F-16C/D. 

Although  a  derivative  development  program,  MSIP  has  incurred  substantial  costs: 
about  $1  billion  to  date.  But  it  has  successfully  handled  the  risks  associated  with 
sophisticated  new  capabilities.  In  particular,  it  has  successfully  managed  the  risks 
associated  with  integrating  many  subsystems  at  once.  Its  ability  to  do  so  allows  it  to  design 
and  implement  new  variationa  on  the  F*16  quidcly.  MSIP  has  successfully  fielded  a  series  of 
effective  F-16C/D  designs.  It  has  also  set  the  stage  for  developing  F-16  variations  outside  the 
F-16(VD  program. 

In  the  end,  MSIP  is  as  much  a  general  approadi  to  tystem  development  as  it  is  a 
formal  F-16  program.  The  F-16  program  has  used  this  general  approach  to  upgrade  the 
F-16A/B  fleet,  extend  the  F-16Cff)  over  time,  and  design  new  F-16  variations  based  on  the 


two  basic  designs.  Such  variations  include  an  enhanced  air  defense  filter  for  the  Air 
National  Guard,  a  new  reconnaissance  aircraft,  an  aircraft  that  emulates  Soviet  fighters  for 
the  Navy,  and  the  potential  for  a  new  aircraft  to  provide  close  air  support.  For  each 
variation,  the  F-16  program  has  selected  subi^stems  like  those  used  in  MSIP  and  integrated 
them  using  methods  similar  to  those  used  in  MSIP.  For  each,  the  F-16  program  generated  a 
design  and  implemented  it  quickly  at  a  reasonable  cost  Such  a  capability  could  prove  useful 
in  the  future  on  a  broader  basis,  outside  the  F-16  program,  to  enhance  the  Air  Force’s  ability 
to  adjust  to  a  changing  world. 
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AFFTC 

AFOTEC 

AFSC 

AFTI 

AGM-65D 
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ALQ-131 
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AMRAAM 
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ASD 

ASPJ 

ATA 

ATDL 

ATF 

ATHS 

BPS 

CARA 

CAS 


GLOSSARY 


A  variant  of  the  F-IGCVD  proposed  as  a  CAS  aircraft. 

Advanced  control  interface  unit,  a  device  developed  as  a  result  of  problems 
identified  during  MSIP  to  facilitate  the  integration  of  the  AMRAAM  with  other 
systems. 

Air  Defense  Fighter,  a  variation  of  the  F-16  developed  rising  an  approach  like 
that  in  MSIP. 

Alternate  Fighter  Engines,  FlOO-FW-220  and  FllO^E-100,  developed  to 
compete  with  one  another  on  a  continuing  basis  as  engines  for  the  F-16  and 
F-15. 

Air  Force  Fli^t  Test  Center  at  Edwards  Air  Force  Base.  Home  of  the  F-16  and 
LANTIRN  combined  test  forces. 

Air  Force  Operational  Test  and  Evaluation  Command. 

Air  Force  Systems  Command,  parent  command  for  the  F-16  SPO. 

Advanced  Filter  Technology  Integration  program,  a  development  program  for 
testing  advanced  subsystems  in  an  F-16  testbed. 

Maverick  air-to-ground  missile. 

Advanced  interference  blanking  unit,  a  device  developed  as  a  result  of  problems 
identified  during  MSIP,  to  mitigate  electromagnetic  incompatibilities  among 
F-16  subsystems. 

An  electronic  waifSaie  pod  considered  as  an  alternative  to  ASPJ. 

The  initial  radar  warning  receiver  considered  for  MSIP.  Later  replaced,  as  a 
result  of  MSn*  activities,  by  the  ALR-56M. 

Advanced  medium  range  air-to-air  missile,  a  critical  element  in  the  Block  40 
upgrade. 

The  fiire-control  radar  used  in  a  series  of  variations  in  MSIP.  It  began  as  the 
improved  or  advanced  APG-66  and  then  evolved  into  the  APG-68M,  a  lower 
cost  version,  and  APG-68V,  a  more  reliable  version. 

Aeronautical  Systems  Division,  the  immediate  parent  organization  for  the  F-16 
SPO. 

Airborne  self-protection  jammer,  a  key  element  of  MSIP  that  never  worked  as 
expected. 

Advanced  terrain  avoidance,  a  capability  developed  during  MSIP  to  enhance 
LANTIRN’s  capability  to  support  low-altitude  fli^t 
Adaptive  target  data  link,  a  system  developed  in  response  to  problems 
identified  during  MSIP  to  integrate  GPS,  PLS8,  and  JSTARS. 

Advanced  tactical  filter. 

Automatic  target  hand-off  system. 

Battery  power  siqiply. 

Combbied  altitude  radar  altimeter,  a  key  subsystem  integrated  during  MSIP. 
dose  air  support,  a  combat  function  served  by  a  series  of  proposed  F-16 
variants. 

Contract«hange  proposal,  the  principal  management  device  used  to  organize 
and  control  new  development  wad  integration  taA*  under  MSIP. 
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CEB 


CNI 

CTF 

DFLCS 

DMT 

DoD 

ECP 


E^GS 

EJS 

EPA 

EPG 

F-16A/B 

F-16ADF 

F-16C/D 

F.16N 

FANG 

FIM 

FUR 

FPI 

FSD 

FSX 

FYDP 

GPS 

HARM 

HUD 

IFF 

IOC 

ISPR 

JPO 

JSPO 

JSTARS 


JTID8 


Common  or  configured  engine  bay,  a  device  developed  as  a  response  to 
problems  identified  during  MSIP,  to  enable  the  F-16C/D  to  accept  easily  eitho- 
AFE  engine. 

Communications/Navigation/Identification  system. 

Combined  test  forces. 

Digital  flii^t  control  ssrstem,  a  critical  qrstem  introduced  during  MSIP  and 
central  to  realization  of  LANTIRN-related  capabilities. 

Dual*mode  transmitter,  a  line  replaceable  unit  in  the  APQ-68  that  experienced 
serious,  unexpected  development  and  produdbility  problems. 

Department  of  Defense. 

Engineering-change  proposal,  the  principal  management  device  used  to 
transform  capabilities  developed  through  CCPs  into  actual  capabilities 
incorporated  on  production  aircraft. 

Enhanced  envelope  gun  mg^t. 

Enhanced  JTIDS. 

Economic  price  ac^ustment  clauses. 

European  Participating  Governments  of  Belgium,  Demnark,  The  Netherlands, 
and  Norway. 

The  first  production  version  of  the  F-16,  upon  which  the  F-16C/D  is  based. 

An  air  defense  version  of  the  F-16  devdoped  as  an  upgrade  of  the  F-16A/B. 

The  version  of  the  F-16  developed  during  MSIP. 

A  version  of  the  F-16  developed  for  the  Navy  to  emulate  Soviet  filters. 

Fast  Action  Negotiating  Group  procedure,  established  CCP  9101  as  means 
of  quickly  definitizing  all  contract  changes  under  $10  miUion. 

Forward  inlet  module,  a  device  developed  as  a  response  to  problems  identified 
in  BdSn*  to  customize  airflow  for  the  engines  used  in  the  F-16. 

Forward-looking  infiwed  sensor,  the  basic  sensor  technology  embodied  in 
LANTIRN. 

Fixed-price  incentive  contract,  the  k^  type  of  contract  fin*  MSIP. 

Full-scale  development. 

A  new  Mitsubishi  fif^ter  derived  in  part  firom  the  F-16C/D. 

Fiscal-year  defense  plan,  a  DoD  document  specifying  expected  future  resource 
flows  for  defense  qrstems  and  activities. 

Gldbal  Positioning  Sinftem,  a  device  introduced  during  MSIP  to  enhance 
navigation  and  other  location-related  activities. 

Hi^-speed  antiradiation  missile. 

Head-up  display,  a  display  introduced  during  MSIP  to  show  FUR  images  and 
data  on  the  status  of  the  aircraft,  its  stores,  and  its  targets. 

Identification,  firiend-or-fiie  qrstem. 

Initial  operational  capabilify. 

Integrated  System  Perfomiance  Responsibilify,  a  contractual  device  assigning 
specific  integratkm  respoosilHlities  to  the  prime  contractor.  General  Dynamics. 
Jdnt  fRogram  MBce. 

Joint  aystem  program  office. 

Joint  surveillance,  target  acquisition,  and  reconnaissance  system,  an  airborne 
radar  sensor  that  could  potentiaDy  communicate  with  an  F-16  throu^  the 
ATDL  devdtqwd  during  MSIP. 

Jdnt  Tactical  Infimnation  Distribution  System. 
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LANTIRN 

LAU-129 

Mcm 

MFTBMA 

MSIP 

NTE 


OCU 

OSD 

PLSS 

PSP 

QA 

ROT&E 

RP.16 

SAFPAR 

SIL 

SOL 

SPO 

TA 

TAG 

TAP 


TBD 

TEMP 

VHP 

VHSIC 

VNS 

WAG 

WAR 

YP 

YPA 

YPC 


Low  Altitude  Navigation  and  Targeting  Infrared  at  Night  aystem,  a  aenacar  and 
laser  designator  around  which  MSIP  Imilt  its  Block  40  design. 

The  final  modular  rail  laundier  developed  for  the  AMRAAM. 

Modular  common  inlet  duct,  a  device  developed  in  response  to  problems 
identified  during  MSIP,  to  fisdlitate  customizing  air  flow  to  the  APE  engines. 
Mean  fli|^t  time  between  maintenance  actions,  a  logistics  measure  relevant  to 
the  development  of  the  APG-68(V). 

Multinational  Staged  Improvement  Program,  the  program  responsible  for 
developing  the  P-16C/D. 

Not  to  exceed,  a  contractual  term  that,  during  the  period  before  the  task  is 
definitized,  defines  the  maximum  amount  that  a  contractor  can  spend  on  the 
task. 

Operational  Gapability  Upgrade,  a  counterpart  to  MSIP  that  develops  and 
implements  upgrade  prcqpmns  for  the  P>16A/B. 

Office  of  the  Secretary  of  Defense. 

Precision  location  strike  fystem,  a  sensor  initially  sc  Med  for  integration 
though  MSIP,  but  later  dropped. 

Programmable  signal  processor,  a  key  line  replaceable  unit  in  the  APG-68  that 
caused  serious  development  problems  during  MSIP. 

Quality  assurance. 

Research,  development,  test,  and  evaluation,  a  DoD  funding  category. 

A  reconnaissance  variation  of  the  F-16  design  to  cany  a  conformal,  centerline 
sensor  pod. 

Secretary  of  the  Air  Force  pix^ram  assessment  report,  a  regular,  periodic 
briefing  to  the  secretary  on  the  state  of  a  program. 

Systems  Integration  Laboratory,  a  development  resource  at  General  Dynamics 
that  has  played  a  key  role  in  MSIP. 

Statute  of  limitations. 

System  program  ofiSce. 

Test  article. 

Tactical  Air  Command,  the  principal  user  of  F-16s  in  the  U.S.  Air  Force  and  the 
principal  incremental  source  of  test  aircraft. 

Test-analyze-fix,  an  iterative  approach  to  development  that  places  heavy 
emphasis  on  repeated  development  of  empirical  measures  on  qrstems  in 
development. 

To  be  determined. 

Test  and  evaluation  mastm-  pkm. 

Very  hi^  firequency. 

Very  hi^speed  integrated  circuits,  an  advanced  form  of  microelectronic 
technology. 

Vehide  navigation  system,  a  subsystem  in  PLSS  that  provided  the  basis  for  the 
ATDL. 

Wide>an|^e  conventional,  a  ^rpe  of  HUD. 

Wide>an|^  raster,  a  ^rpe  of  HUD. 

F-16  SPO. 

F-16  direcUnate  of  acquisitioa  planning,  dosed  during  MSIP. 

F-16  direetorate  of  configuration  management. 
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YFD  F*16  directorate  of  deployment  and  teat  and  later  development  and  integration: 

under  the  latter  name,  responsible  for  managing  MSIP. 

YPE  F-16direetinrateofengineering,  the  principal  functional  office  supporting 

MSIP. 

YPF  F-lddireetorateoftest  and  deployment,  created  during  MSIP. 

YFK  F>16  directorate  of  contracting. 

YPL  F-16  directorate  of  logistics  planning  and  later,  when  int^rated  with  VP  A,  of 

logistica. 

YPM  F*16  directorate  ofmanufiicturing  and  later  of  manufacturing^qualit. 

assurance. 

YPO  F*16  directorate  of  operations  management 

YPP  F*16  directorate  of  program  control. 

YPR  F-16  directorate  of  projects  and  later  of  development  programs;  under  the  latter 

nnwiA,  responsible  for  managiitg  MSIP  until  integrated  into  YPD. 

YPS  F-16  directorate  of  system  safisty. 

YFX  F-16  directorate  of  multinational  programs. 
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1.  IHTROOUCnON 


Sisk  management  is  a  central  part  of  programs  to  «tovdop  new  tedindogy.  By  its  very 
nature,  the  devd^^nnent  of  new  technology  requires  identificatiQin  and  reduction  of 
uncertainties  about  a  product’s  performance  to  the  point  where  the  product  can  be  successful 
in  actual  use.  Potential  risks  stem  fron,  anumg  other  things,  the  nature  ci  the  continuing 
demand  for  the  services  provided  by  the  new  isroduct,  the  ability  of  the  product  to  provide 
such  services  when  it  is  mature  enou^  to  produce,  and  the  ability  of  developers  to  achieve 
timely  and  cost-effective  system  maturity. 

For  weapon  system  developments,  risk  management  can  be  approached  in  many  ways. 
A  dorivative  approach  allows  a  weapon  system  to  evolve  over  time,  incorporating  new 
technological  capabilities  as  they  present  themselves  or  as  new  system  requirements  emerge. 
Sudi  an  sqpproach  limits  the  ‘Ijumps”  that  can  occur  in  capabilities,  but  it  also  enables  fairly 
quick,  low-risk  adoption  of  changes  once  it  becomes  dear  that  change  is  desirable.  By 
limiting  risk,  it  also  limits  the  cost  of  devdoping  such  an  improvement.  Whereasa 
mainstream  devdopment  mi^t  easily  require  10  to  15  years  to  move  from  a  concept  to  an 
operable  system  in  the  fidd  and  cost  billions  of  dollars,  a  derivative  develoi»nent  can  fidd 
sdected  capabilities  in  a  fraction  of  that  time  for  a  fractitm  of  that  cost 

A  devdopment  approach  that  can  respond  qui^ty  to  dumge,  for  a  reasonable  cost,  is 
likely  to  become  more  important  in  the  near  future.  Dedining  red  defense  budgets  will  limit 
the  resources  avaflaUe  to  devdop  and  produce  new  weapon  systems.  And  tke  exact  nature  of 
the  threat  will  be  more  dusive  than  it  has  been.  Hie  threat  is  hkdy  to  diange  over  time  as 
the  sttuation  dianges.  Because  sudi  a  devdopment  approach  is  designed  to  ded  with 
continuing  diange,  it  is  also  likely  to  offer  attractive,  more  generd  insights  into  risk 
management  in  the  development  of  new  techndogies. 

VnXh  these  perspectives  in  mind,  this  Note  examinee  tiw  apfnoadi  used  to  upgrade  the 
F-16  fighter  over  time  as  new  tedindogicd  capabilities  have  become  available  and  as  new 
threats  have  presented  tbemadves.  The  multimission,  multinationd  the  F-16 

nghtang  Paloon,  comideted  its  initid  fhn-scale-devdopment  (FSD)  effort  and  achieved  iwitiiil 
oparationd  oqiability  (IOC)  in  the  U.S.  Air  Force  in  1979.^  Since  tlw«,  tins  Qenerd 
Dynamics  aircraft  program  has  continued  to  upgrade  the  F-16  and  has  used  it  as  the  basis 
for  a  serfasofmoddsqiecially  tailored  to  a  variety  of  separate  missions  and  the  needs  of  19 

lUB.  Afar  floves,  Amnantfed  Sjrsteais  XMvfeioii,  F-16  Eastern  Program  Offloo,  "Management 
InfenaatiaBNotaboak,”tyri^Patteiaon  Air  Force  Base,  (Mo,  SO  April  1984. 


-2- 


individual  mttiims.  As  a  result,  the  program  hu  prospered.  By  1990,  it  was  amunitted  to 
produdng  almost  2,500  aircraft  for  the  U.S.  Air  Force  and  Navy  and  1,300  aircraft  for  other 
governments.^ 

Much  of  the  aircraft's  eontjnning  success  can  be  attributed  spedficaUy  to  its  ability  to 
mature  over  time  as  teduudogies  and  threats  dumge.  Over  the  last  decade.  General 
Qynamks  has  performed  a  variety  of  devdopment  activities  that  mi»iiitiiiw  the  flexibility  of 
this  filter  and  continue  its  maturation.  This  Note  examines  the  central  development 
activity  during  the  1960s,  the  Multinational  Staged  Improvement  Program  (MSIP).  This 
activity  is  responsilde  inimarily  for  developing  the  modifications  of  the  F'16A/B  that  led  to 
the  hii^er-performance  F*16C/D  in  the  mid-1980s.*  It  continues  to  thin  day  as  more  capable 
F-16G/DS  emerge  from  devdopmenl 

The  ap|»poadi  to  managing  MSIP  used  by  the  U.S.  Air  Force  and  General  Dynamics 
iUnstrates  thdr  more  general  approadi  to  developing  derivative  aircraft  frtmi  an  estaidished 
and  successful  design.  Hence,  it  offiers  lessons  about  how  sudr  development  mi^ht  occur 
elsewhere  in  the  ftitnre.  This  Note  focuses  on  a  development  program  that  has  generally 
been  viewed  as  hi^ily  successful  in  the  hope  that  it  can  offer  lessons  for  managing  the  risks 
of  eystem  development  as  the  world  situation  continues  to  diange. 

THE  M6  MULTINATIONAL  STAGED  IMPROVEMENT  PROGRAM 

To  begin,  let  us  review  briefly  what  M^P  is.  Section  5  will  return  to  this  question  in 
greater  detaiL  MSIP  is  a  development  {dan  managed  by  the  U.S.  Air  Force,  Aeronautical 
Systems  Division  F-16  System  Program  Office  (F-16  SPO),  and  General  Dynamics  to 
oomthnate  improvements  added  to  the  F-16  over  time.  It  began  foimaUy  in  1980  as  an  effort 
to  sinichronixe  the  introduction  of  a  series  of  enhancements  to  the  P-16  as  new  capabilities 
became  available  to  be  incorporated  on  the  platform.  At  that  time,  an  F-16  SPO  official  said: 

looking  ahead,  two  naior  foctoca  will  dominate  the  fotoie  oourae  of  the  F-16  program: 

MSIP  and  the  reahsatiim  of  fineign  military  aalae.  Thaaa  two  foetesdiractiy  impinge  on 

the  length  of  the  pndoctian  ran  and  therriy  determine  the  lifespan  of  the  F-16 

program.* 


NSenanl  Oynaatiee,  1990. 

*Ihe  A  and  C  models  hove  only  one  pilot  Ihe  B  and  D  modrie  fecilitate  traiidng  by  providing 
[^oeeftrtwopllats.  Otiiarwlso,  the  A  and  Bmodele  are  very  dose  to  being  fonetionally  equivalent  as 
an  the  C  and  D  andds. 

*UB.  Air  Fime,  Asronaatieal  Systems  Divisioa  F-16  SPO,  ‘^Semianaaol  Hiatorical  Rep«t  1 
Ja|y-81  Oseambar  1990,*  Writtitt-Pattenoo  Air  Force  Boas,  Ohio. 
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TwooonnderadfliuwaMimiMrtastmthedaiRgnofliSIP:  eompakifaility  of 
c«|MJwlitiM  and  planning  ferpotantialagiriyitMat.  Hm  firat  eonaidantieo,  to  anaitra  that 
new  capabiUtieg  ware  compatihle  with  one  another,  required  onmpatihiKty  «hadw  at  all 
levtda.  At  the  ^pleat  lewd,  all  compaoento  antidpatod  on  tiie  aircraft  had  to  fit  and  had  to 
be  compatilde  with  the  aiiframa's  baatc  aorodynaiBiea.  Computers,  radara,janin>era,  and 
weapons  had  to  woric  together  to  ensure  thairdbelireineorporatioa  on  new  weaponry.  New 
components  also  had  to  avmd  interfering  with  one  another's  alectromagnetie  frequencies. 
Codqnt  controls  and  displnys  were  needed  for  eadi  new  capability;  even  as  the  aircraft 
haramn  MwnpUw,  eontods  and  disidaya  had  to  remain  as  simple  as  possible  to  avmd 
overwhelming  the  aircrafts  sin^e  pilot  And  the  aircraft  had  to  provide  adequate  power  and 
environmental-oontrd  services  to  support  all  new  systems  and  adequate  thrust  to  lift  those 
new  systems  into  omnbat. 

Second,  installation  of  the  structure  and  wiring  to  support  the  new  systems  could  be 
<«tM>ap«w  if  it  were  done  during  the  manu&cture  of  an  airframe  ratlmr  than  being  retrofitted 
when  a  capability  became  available.  This  insist  suggested  introducing  so-called  Group  A 
provisions  on  new  aircraft,  which  were  basically  wiring  and  structures  to  support  Group  B 
hardware  and  sdtware  that  would  be  installed  in  new  aircraft  and  retrofit  into  aircraft  with 
the  appnqniate  Group  A  fittings.  Designers  had  to  weigh  against  the  potential  savings  the 
possibility  thet,  far  tedinical  or  budget  reiuMms,  Group  B  add-ons  m4dit  not  occur  in  the 
future  and,  even  if  thoy  did,  the  added  weight  of  Group  A  provisiong  would  impede  aircraft 
performance  until  Groiq>  B  provisions  were  added. 

That  is,  MSIP  has  essentialty  been  a  develoimient  program  aimed  at  incorporating 
many  diqMuratecaqmbilitiesinaoidiermitway.  Whereas  development  as  sq;>arate,  modular 
aytoems  is  the  key  to  the  new  capab»litieg,intogratingsodi  modules  and  incorporating  than 
into  the  production  of  new  F-16a  to  realize  their  capabilities  is  the  key  to  BISIP.  Although 
MSIP  ptomung  and  testing  activities  for  a  partieolar  componmit  typically  occur  in  parallel 
with  the  ftdl  ecaledevdopment  and  final  product  verification  for  that  component,  MSIP 
remainadistiiietflrom  tile  latter,  subayatem-apedficactivitioB.  ki^fiieuaes  on  integration. 
Even  when  integration  activities  reveal  the  need  far  new  capabilitieg  in  a  cmnponent  or  for 
new  components,  devdopment  activities  relevant  to  subqystsm-qMdfic  activitiea  gmeralty 
remain  aeparato.  DistinguiahinglCSIP  from  the  individoal  development  imgrama  is  dton 
difliciilt,  partiodaity  whan  the  F-16  8PO  provides  aircraft  as  testbeda  for  new  toduMdogies 
and  other  inputs  into  individoal  davdo|«ients  meant  to  enhance  integration  in  the  future. 
But  the  distinction  is  important  and  real. 
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Tbe  diiitinctiim  u  clarified  1^  atating  that  the  F-16  SPO  and  Genmd  Dsnamics 
organize  their  integration  aetiTitiea  around  blocks  of  new  aircraft.  Each  Mock  of  aircraft  has 
an  identifiaUe  constellation  of  qrstems  that  must  be  integrated.  Blocks  5, 10.  and  15 
involved  improvaments  in  the  reliability,  supportability,  and  produdbility  of  the  F-16A/B 
design.  Blocks  25, 30/32, 40/42, 50/52,  and  higher  invtdved  extensive  enoufldi  changes  to  call 
for  a  designation  diange  firom  F*16A/B  to  F-IOC/D.^  The  systenu  to  be  included  in  a  Uock 
change  over  time  as  more  information  acromulates  about  their  availability  and  capability. 
The  F>16  SPO  and  General  Itynamics  have  created  an  extremely  flexible  managemoit 
environment  in  which  contractchange  pnqMsals  constantly  a4iust  the  MSIP  activities 
associated  with  new  systems,  and  engineering-change  proposals  incorporate  the  new  s}'stems 
in  new  F-16e  mr  retn^t  them  in  existing  F-16e  as  MSIP  completes  its  integration  tasks. 

MSIP  conceived  blocks  finm  Block  15  and  hi^er  as  part  of  a  three-stage  program, 
accounting  for  the  name,  MSIP.  Table  1.1  summarizes  these  stages  as  they  were  described 
in  1990;  as  noted  above,  thty  changed  somewhat  as  MSIP  proceeded. 

Stage  I  required  little  development  or  design  and  called  primarily  for  the  installation 
of  new  structures  and  wiring  in  Blodr  15.  Sudi  installation  was  achieved  primarily  throu^ 
a  singde  engineering-change  proposal.  The  second  and  third  stages  required  new  contracts, 
and  maoy  contract- and  engineeting-change  proposals.  Stage  II  incorporated  improved  radar 
and  engines,  enhanced  munition  capabilities,  and  power  and  cooling  capabilities  to 
accommodate  future  dianges  in  Blo^  25  and  30/32.  This  stage  initiated  the  devdopment  of 
the  F-16C/D  per  se.  Stage  III  built  a  new  kind  of  fi|d*ter  around  the  n4d)t/all-weather 
ciqmbilities  allowed  by  lANTIRN  in  Blodc  40/42.  It  added  many  advances  in  avionics  and 
new  engines  in  Block  60/52,  which  is  not  to  use  the  LANTIBN  system.  As  new  capabilities 
are  added  to  new  blocks  of  productkm,  numy  will  also  be  retrofitted  into  blodu  of  ariating 
aircraft. 

lliese  stages  are  useful  as  planning  ccmstaructs;  however,  administration  of  the 
dmnges  made  possible  throu^  MSIP  has  focused  on  the  specific  blocks  of  aircraft  involved. 
As  a  result,  for  most  of  Hs  histosy,  MSIP  has  included  activities  managed  by  more  than  one 
program  office  in  the  F-16  SPO.  And  budgets  that  distinguiah  MSIP  from  non-MSIP 
aetivitias  are  difficult  to  dffins. 

*Blo^  80  and  83  difhr  only  in  thefar  anginas:  Whereas  BloA  30  usm  the  General  Electric 
F110-GK-100eiigiiie.BioA82aseetheiyattandWhitnsyF100-PW-S20engiiie.  Hw  same  ivpte  to 
Blo^  40/43  and  50^.  BloA  names  duaged  over  the  oonise  of  MSIP.  For  exaiimle,BlockB  40/42  and 
60/sa  WHS  inttially  known  as  Blo^  300  and  80P. 
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Tabl*l.l 

B— ic  8t«gM  of  tlf  MwItifmtkwuJ  Staged  Improvement  ProgrMi 


Stage  (Date*) 

Block 

Maior  Changes  Induded 

Stage  I  (1960) 

Block  15 

Structto*  and  wiring  piovisiana  for  future  systems 
Inereased-aiea  horuantal  tail 

Stage  n  (1981) 

Block  25 

AGM-65D  Maverick  APG-68  Sre-contrcd  radar 

Enhanced  avionics  and  codqut 

Wide-angto  conventional  he^up  display  (HUD) 
Inereaaed-capacity  electrical  power  and  cooling 

Block  30/32 

AMRAAM  provisions 

Shrike 

Alternate  Fi^itor  Engines  (AFE)  with  configured  engine 
bay  (CEB) 

Memory  expansion 

Seal-bo^  tanks 

Stage  m  (1985) 

Blod(  40/42 

Low  Altitude  Navigation  and  Targeting  Infrared  for  Ni^it 
(LANTIRN)pods 

LANTIRNHUD 

(Robal  Poeitioning  System  (OPS) 

Hidi-QMod  Antiradiation  Missile  (HARM)  n 

APG-68V  fire  control  radar 

Expanded  computers 

Distal  flight  control  aystem  (DFLCS) 

Automatic  Terrain  Following 

Block  50/52 

Additumal  changes  in  weapima,  radars  and  other  avionics, 
ooc^nt,  engines,  and  reliability  otMi  maiiitiiiimhility 

SOURCE:  General  Djmamica,  1990,  pp.  88, 86. 

‘Data  when  deealopnMnt  and  integration  for  the  etage  began. 


Throns^umt  MSIP,  the  U.S.  Air  Force  has  coordmated  devdopment,  integratitm,  and 
production  incorporation  planning  with  the  so-called  European  Participating  Governments 
(EPG)  of  Belgium,  Denmazfc,  Hm  Netherlands,  and  Norway,  which  were  involved  in  the 
multinational  F-16A/B  program.  A  quidc  review  of  the  rdative  numbers  of  aircraft  procured 
by  these  governments  (527)  and  the  United  States  (2485)  reveals  that  tlie  United  States  has 
been  the  dominant  partner  in  this  effiat,  ediidi  has  been  true  throu^out  MSIP. 
Nonetheless,  MSB*  has  proceeded  on  the  understanding  that  the  participating  governments 
would  probably  want  access  to  the  capabilities  being  devdoped  and  that  much  of  the  woric 
required  to  produce  the  F-16G^  resulting  from  MSIP,  would  in  fitet  be  peiftnaed  by 
companies  located  in  those  countries.  Hence,  ahhongh  U.S.  Air  Force  priorities  have 
dominated  MSIP  from  the  beginning,  multinatiopal  participation  in  the  program  has  also 
been  important 

What  began  as  a  formal  three  stage  |dan  has,  more  and  more,  become  viewad  as 
dBBOst  aTnonymons  with  continuing  development  effivts  for  tlm  F*16,  more  an  apinoach  to 
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development,  integration,  and  production  incorporation  than  a  formal  program  in  its  own 
right.  So  it  is  n<rt  surprising  that  when  the  Air  Force  began  to  omtemplate  further  changes 
radical  enou|^  to  call  for  another  designation  change  to  an  F-16E  firom  an  F-16C/D,  many 
began  to  refer  to  the  development  program  for  the  Agile  Falcon  follow-on  to  the  F-16C/D  as 
MSIP IV,  the  fourth  stage  of  the  continuing  develoianent  program.  As  events  nnfolilftri, 
efforts  to  initiate  this  new  activity  foiled. 

But  MSn*  itself  continues  as  the  F-16C/D  continues  to  mature  in  new  capabilities  that 
can  be  installed  on  new  aircraft  or  retrofit  on  ensting  F-16G/Ds.  Viewed  more  broadly,  MSIP 
epitomises  the  development  of  other  aircraft  derived  from  the  F-16,  efforts  to  retrofit  wTiating 
F-1€A/Bs  to  incorporate  new  capabilities,  and  efforts  to  customize  aircraft  for  the  needs  of  the 
many  non-U.S.  governments  that  continue  to  buy  new  F-16A/Bs  and  F-lSC/Ds. 

ANALYTIC  APPROACH 

MSIP,  then,  is  a  structured  means  of  coordinating  the  introduction  of  many  new 
capabilities— the  general  approach  itself,  a  top-down  entity  that  transcends  the  individual 
components  managed  by  MSIP.  Viewed  from  the  bottom  up,  MSIP  is  simply  the  sum  of  the 
mjnriad  improvements.  We  can  understand  the  success  of  MSIP  only  by  understanding  each 
improvement  and  its  incorporation  in  the  F-16  throu^  MSIP.  In  fact,  both  top-down  nnH 
bottom-up  perspectives  are  vaUd;  th^  simply  offer  different  wasrs  to  look  at  the  way  MSIP 
works.  Both  perspectives  are  reflected  in  this  document.  The  material  presented  here  is 
based  primarily  on  management  documents  and  historical  reports  prepared  by  General 
Dynamics  and  the  F-16  SPO  during  the  period  and  on  interviews  with  individuals  aanncigtH 
with  MSIP  daring  the  IdSOs. 

The  Note  focuses  almost  entirely  on  events  and  drcumstances  within  the  program 
and,  in  particalar,  on  events  and  drcumstances  relevant  to  U.S.  versions  of  the  F-16. 

Farther  work  is  needed  to  examine  the  external  circumstances— in  the  Air  Force  and  the 
contractors  most  directly  involved  in  the  development— in  whidi  thi«  development  proceeded 
the  international  dimensions  of  the  development;  time  and  resource  constraints  did  not 
permit  us  to  eaamine  these  topics  carefully  in  this  study. 

FoUowing  a  brief  description  of  our  ^iproach  to  risk  nd  risk  managemmit,  in  Section 
2,  Section  s  deacribes  the  F-16  SPO  and  the  SPCs  management  of  MSIP  over 

pastlOyeaza.  As  MfflP  proceeded,  the  SPO  reorganised  to  reflect  the  growing  maturity  of 
the  F-16G/D;  Sectfam  3  discasaes  these  diangBS  as  wdL  Saetian  4  a»plAii>B  in  gmatay  Hwtaiil 
the  F-16  devefopment  program  during  the  1980s,  when  MSIP  was  active.  It  emphasizes  that 
operated  in  a  broader  aetting  and  that  it  exemplifies  the  type  of  development  activities 
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occurring  elaewhore  in  the  SPO.  Then  Section  6  deacribes  MSIP  itaelf  in  greeter  detail, 
explaining  the  m«^r  technological  changes  it  has  efiSBcted,  the  managonent  strategy  it  has 
used  to  effect  those  changes,  and  the  contracts  used  by  tiie  F-16  SPO  and  Gennral  Pynamics 
to  govern  their  rcdatiimship  during  MSIP.  These  sections  provide  what  is  essnntislly  a  top- 
down  discussion  of  MSIP,  treating  it  as  a  unified  process  that  is  greater  than  the  sum  of  its 
parts. 

The  Appendix  adopts  a  view  from  the  bottom  by  describing  six  subsystems  introduced 
in  MSn*  II  and  III  and  using  them  to  illustrate  important  aspects  of  MSIP  and  its 
management  of  risk.  Table  1.2  summarizes  pertinent  information  about  these  subsystems. 
Each  subsystem  has  a  distinctive,  and  sometimes  turbulent,  development  history  separate 
firom  MSIP.^  Each  system  was  prominent  in  the  concerns  of  the  managers  responsible  for 
MSIP.  As  Table  1.2  shows,  the  subsystems  also  span  a  range  of  factors  relevant  to  IdSIP: 
the  major  functional  capabilities  represented  in  MSIP — cockpit,  avionics,  munitions,  and 
other  major  components — and  three  blocks  of  MSIP  Stages  II  and  m,  changes  in  two 
components  across  blocks  help  illustrate  the  degree  of  flexibilily  in  MSIP.  The  F-16  SPO 
oversaw  the  development  of  some  but  not  most  of  the  changes.  All  but  one  are  important 


TaMel-S 

Selected  Sabeystems  Studied  in  Greater  Detail 


Subeystem 

l^pe  of  System 

Block 

Introduced 

Developing  SPO 

Government  or 
Contractor 
Furnished 

AFG-68  fire-control 
radar 

Avionics 

26, 40" 

P-16 

Government 

AMRAAM  and 
launcher 

Muniticn 

30 

AMRAAM,  F-16^ 

Government 

LANTIRNpods 

External  avionics 

40 

LANTIRN 

Government 

Head-up  display 
(HUD) 

Codqpit  display 

30,40* 

P-16 

Contractor^ 

CHobal  Positioning 
System  (GPS) 

Avionics 

40 

GPS 

Government 

AUemate  Flatter 
Engine  (AFE) 

Propulsion 

30 

Propulsioa, 

F-16* 

Government 

*laipra««d  veraion  (V)  of  AP048  tatrateetd  at  Block  40. 

^•16  Sro  ovonaw  ladaoifn  af  laandwr. 

*CVD  HUD  intndnead  at  Blodt  80,  LANTIRN  HUD  at  BloA  40. 
^Chaastd  to  eoetractor  ftiniiaiiad  dnrias  MSIP  to  IlKilitatB  iotifratioa. 
•P.iegPOiiiMMwiU— UuMMutaf..—— 


*For  mote  detail  on  the  AMRAAM,  LANTIBM,  OPS,  and  mgiea  programe,  eee  this  Note’e 
companion  caee  etudiee. 
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enougfa  to  be  government  furnished.  The  exception  was  changed  from  govemmmit-  to 
contracter'inovided  equipment  to  pranote  the  integration  task  of  MSIP.  In  sum,  we  cannot 
say  that  these  examples  tell  all  of  the  stories  important  to  risk  management  in  MSIP,  but 
they  provide  useful  illustrations  that  relate  to  many  of  the  MSIP-affected  factors. 

Section  6  closes  the  Note  with  general  policy  conclusions  and  suggestions  for  future 

work. 
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2.  RISK  ASSESSMENT  AND  RISK  MANAGEMENT 


Weapon  system  development  is  an  inherently  risky  activity — a  statement  with  which 
many  defense  personnel  and  contractors  would  agree  but  the  precise  meaning  of  which  would 
be  difficult  to  agree  on.  Most  would  concur  that  risky  connotes  that  system  development  is 
not  a  predicts'  >le  process,  and  that  the  activity  involves  many  surprises,  or  events  with 
negative  outcomes.  Thai;  is,  the  word  risk  suggests  not  only  unpredictability,  but  danger. 
This  definition  becomes  especially  true  when  we  discuss  not  just  risk  but  risk  management. 
Those  who  manage  risk  have  a  distinct  desire  to  ameliorate  the  negative  effects  associated 
with  the  unpredictability  of  a  weapon  Qrstem  development. 

If  we  accept  this  view,  many  ways  remain  to  define  precisely  what  risk  is.  The  very 
unpredictable  nature  of  risk  itself,  however,  tends  to  defy  finder  formalization.  Any  attempt 
to  be  precise  about  what  risk  is  tends  to  give  up  some  aspect  of  unpredictability.  It  is 
difficult — and  perhaps  even  misleading— to  characterize  too  precisely  a  situation  about  which 
we  are  profoundly  uncertain.  That  said,  analysis  benefits  firom  precision.  This  section  briefly 
examines  the  view  by  development  managers  of  risk,  risk  assessment,  and  risk  management 
and  defines  these  concepts  to  order  our  inquiry  in  the  sections  that  follow.^ 

A  REALISTIC  WAY  TO  THINK  ABOUT  RISK  IN  ANALYSIS 

The  predominant  analytic  definition  of  risk  is  probably  that  of  economists  and  decision 
theorists,  which  emphasizes  unpredictabihfy.  For  economists,  risk  or  uncertainty  exists 
whenever  unpredictalnlity  exists.’  Ridt  associated  with  a  process  increases  as  the  range  of 
possible  outcomes  of  that  process  increases.  More  formally,  risk  increases  as  the  variance  of 
outcomes  associated  with  the  process  increases.  To  illustrate,  consider  the  two  distributions 
in  Figure  2.1.  The  outcome  of  a  process  is  represented  on  the  horizontal  axis  in  terms  of  a 
sin|^  metric  of  performance.  Subjective  probabilify  density  lies  on  the  vertical  axis.  Based 
on  this  definition,  distribution  Dl  is  riskier  than  distribution  D2  because  D1  is  more  diffuse 

shditly  revised  verakn  of  this  sectioa  appears  as  Sectioa  2  in  Camm,  1993. 

’After  thio  wo^  was  complete,  Steven  Garber  of  RAND  broudit  the  fidlowing  refemwes  to  my 
attention.  They  oonSrm  that  the  views  of  risk  aaBeaaaBaot  and  risk  aaanagoaaent  that  we  observed  in 
the  Air  Faroe  are  oonaistait  with  those  observed  mote  bnadfy  in  private  industry.  Cf.  Garber, 
forthoominc  MaeCrinunon  and  Wehmng,  1966;  Mardi  and  Shapira,  1987;  and  Sbeqnn,  1986. 

tylMiy  economists  vronld  go  ftgtfaer  to  distingoiahrtiAihimBnoertewity.  Ri^  occurs  when  the 
unpredietdiility  is  asaodatodwifo  the  outcomes  ofawell'understoodstodiastic  process;  uncertainty 
occurs  when  m^redictahfltty  results  than  outcomaeofapooriy  understood  proceee.  Arelated 
distinethm  will  be  uasftil  to  us  below. 
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Ficon  2.1— Sobjective  Probability  Denaity  Diatributiona  D1  and  D2  for  Two  Programa 


than  D2.  D1  is  riskier  even  thou^  the  central  tendency  for  Dl  is  well  above  that  for  D2  and 
would  be  riskier  even  if  Dl  stochastically  strictly  dominated  D2.* 

Now  suppose  that  Dl  and  D2  represent  the  expected  outcomes  of  two  different 
approaches  to  developing  a  weapon  system.  The  metric  of  performance  might  be  the 
probability  that  a  filter  aircraft  prevails  in  a  standardized  air-to-air  engagement  with  the 
enemy.  Viewing  these  alternatives,  weapon  ^ratem  developers  would  agree  that  D2 
represents  the  riskier  approach.  They  would  justify  this  position  by  pointing  out  that  poor 
outcomes  are  more  likely  with  D2  than  with  Dl.  Going  further,  some  mi^t  be  willing  to  set 
a  minimiiin  standard  probability  of  success  S  fm*  the  aircraft  and  characterize  risk  as  the 
subjective  probability  associated  with  outcomes  lower  than  this  standard.  For  example,  if 
the  standard  were  S  in  Figure  2.2,  which  re-creates  the  distributions  in  Figure  2.1,  the  risk 
associated  with  each  alternative  would  be  proportional  to  the  shaded  areas  R1  and  R2, 
representing  the  subjective  probabilities  that  the  aircraft  designed  by  eadi  process  failed  to 
meet  the  set  standard. 

^Ibat  is,  suppose  that  we  imagine  random  draws  firom  both  distribations  simultaneously.  If  we 
believe  that  ontoomes  for  the  two  disteibutions  are  correlated  so  that  the  outcome  for  Dl  always 
dominates  that  for  D2,  then  Dl  stochastically  strictly  dominates  D2. 


s 

Metric  of  performance 

Figare  2.2— Bukfl  R1  and  R2  Aaaoeiatttd  with  Two  Weapon  Sjwtem  Development  Programs 
and  Oeenrring  Bdow  the  Set  Standard  for  Performanee  S 

Devdrqiment  managers  would  find  this  view  of  their  decision  environment  grossly 
oversimplified.  For  example,  such  managers  do  not  generally  attempt  to  estimate,  even 
approximately,  the  probability  of  failure  as  defined  above  or  to  compare  such  estimates 
across  policy  alternatives.  Understanding  this,  we  can  still  use  the  approadi  offered  in 
Figures  2.1  and  2.2  to  provide  a  useful  metaphor  for  fliinking  about  decisionmaking  in 
we^wn  system  development  Let  us  continue  with  this  metaphor  for  a  moment  before 
returning  to  the  question  of  how  precise  managers’  views  of  risk  mi^t  be. 

The  density  functions  in  Figures  2.1  and  2.2  are  essentially  risk  assessments.  Ridr 
managers  cannot  effectively  make  sudi  assessments  indqwndently  of  the  policies  they 
intend  to  use  to  manage  risk.  T1iatis,theyeffectively  viewridtmoaoigemenf  asa  way  to 
alter  the  shape  efthedistiibutioiis  shown.  At  any  point  during  a  development  we  can  think 
of  the  manager’s  subjective  beHeft  about  the  program’s  outcome.  Such  beliefs  change 
throu^  the  course  of  a  development  If  the  manager  expects  success  at  a  certain  point  in 
time,  he  or  she  has  adopted  polidos  that  restrict  the  degree  of  risk  associated  with  such  areas 
as  R1  and  K2  to  an  acceptably  low  level.  Some  of  those  policies,  sudi  as  an  acquisition  plan, 
cystem  q>ecifications,  contract,  or  tMt  plan,  can  be  established  by  the  manager  today.  Some 
ofthem  cannot  be  made  e^lidt  in  advance.  The  manager  must  expect  surprises,  the  details 
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of  which  cannot  be  known  and  planned  for  in  advance.  Each  such  surprise  will  presumably 
alter  the  manager’s  risk  assessment  and  force  a  change  in  policy  in  some  way  to  get  risk 
under  control  again. 

Viewed  in  this  way,  risk  mani^ment  begins  to  look  very  much  like  the  general 
maimgement  of  a  developmoit  program.  And,  in  fact,  devdopmmit  managma  draw  little 
distinction  between  the  two.  In  a  sense,  the  central  task  of  a  development  program  is  to 
ftliminatft  basic  uncertainties  about  a  new  design  so  that  it  can  be  transformed  into  a  useful 
product.  Doing  so  takes  time,  introducing  risks  associated  with  the  enviitmment  in  which 
development  occurs  and  in  which  the  product  will  be  used.  Development  managers  are  quite 
comfortable  thinking  about  development  in  these  terms,  bringing  risk  management  per  se 
dose  to  their  core  concerns  in  the  course  of  a  development. 

That  said,  risk  management— or  more  generally,  program  management— ^or  a 
development  is  much  more  complex  than  the  simple  metaphor  above  would  suggest 
Managers  do  not  generally  think  in  terms  of  subjective  probability  densities  such  as  those 
presented  above.  They  think  more  in  terms  of  contingendes:  What  would  hasten  if  lAis 
happened?  Rou^y,  how  likely  is  it?  What  kind  of  trouble  would  it  cause?  What  can  I  do 
now  to  mitigate  that  trouble?  What  kind  of  resources  or  staff  would  I  want  then  to  deal  with 
it?  This  process  of  assessing  risk,  planning  for  it,  and  reacting  to  it  is  what  we  want  to 
understand  better  in  this  Note.  The  metaphor  above  helps  us  understand  that  managers 
generally  make  such  assessments  by  focusing  on  surprises  that  can  hurt  them  and  seddng 
ways  to  mitigate  the  effects  of  those  surprises. 

A  SIMPLE  STRUCTURE  FOR  INQUIRY 

Surprises  come  from  a  variety  of  sources.  They  affect  a  development  isogram  in  many 
wa3rs.  And  managers  have  a  number  of  tools  for  planning  for  and  responding  to  surprises. 

SOUTOMOf  RMc 

Managers  lode  for  surprises  in  two  places:  outside  the  development  and  within  it. 
First,  develoinnait  takes  time.  While  it  occurs,  the  world  outside  the  development  can 
diange,  predpitating  surprises  for  a  development  program.  Most  basically,  changes  in  the 
threat  can  alfoct  either  willingness  to  continue  frmding  the  program  or  the  requirements  set 
for  the  final  product  Changes  in  technology  can  alfect  the  availability  of  subsystem 
capabilities  rdied  on  by  the  devd<qnnent  m  the  need  for  tire  system  under  devdopment. 
Changes  in  the  economy  can  modity  the  cost  of  the  develtyiment  itself,  that  of  the  final 
product,  or  the  availability  of  funds  to  maintain  the  development  as  expected.  Changes  in 
the  Air  Force  testing-and-evahiation  community  can  affect  the  availability  rftest  assets.  All 
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these  foeton  are  eeaentially  beyimd  the  manager’s  control.  However,  he  or  she  can  reduce 
their  ^eets,  generally  by  restricting  the  length  of  a  development,  so  that  fewer  opportunities 
for  surprises  arise  over  the  course  of  the  development.  More  likely,  the  manager  must 
anticipate  specific  types  of  surprises  and  tailor  individual  responses  to  each  type. 

Second,  evm  if  the  world  outside  the  development  remains  stable,  surprises  can  be 
erpagted  fiem  within  the  development  Ezamides  are  development  efiorts  that  require  more 
timft  or  resources  than  expected  to  reach  a  particular  performance  improvement  and  certain 
fawhnirail  goals  Set  in  the  program  that  turn  out  to  be  infeasible.  The  manager  has  greater 
control  over  such  factors,  but  can  still  not  expect  to  eliminate  such  surprises. 

Program  Attributaa  Affedad  by  Risk 

When  surprises  occur,  they  can  affect  a  number  of  program  attributes.  First  and 
foremost,  they  can  afEsct  the  probability  that  the  program  will  survive  to  yield  a  useful 
product.  Assuming  successful  ptt^ram  completion,  th^  can  affect  the  resources  and  time 
required  to  complete  the  program;  these  are  the  “cost”  and  “schedule*  criteria  normally 
associated  with  devdopment.  Surprises  can  also  afEsct  various  measures  of  final  tystmn 
“perfiirmanoe.*  Sudi  logistics-oriented  factors  as  reliability,  availability,  maintainability, 
and  operating  and  support  costs  are  increasingiy  considered  important  parts  of  system 
performance.  Traditional  measures  of  system  performance  emphasize  combat  c^wbility  and 
can  normally  be  measured  in  a  variety  of  wa3rs  specific  to  each  system.  Producibility  and 
production  cost  for  the  syston  round  out  the  performance  factors  rdevant  to  the  manager. 

As  a  development  program  is  normally  defined,  a  manager  will  have  a  hard  time 
meeting  his  or  her  goals  on  all  of  the  above  fisctors.  To  increase  the  probability  of  program 
survival  eariy  in  the  program’s  life,  the  manager  must  make  the  program  look  attractive 
relative  to  alternative  programs.  Hence,  the  manager  generally  attempts  to  understate  goals 
for  development  cost  and  schedule  and  overstate  the  performance  goals  of  the  system.  To  the 
extent  that  such  goals  are  adopted  as  standards  like  those  in  Figures.  2.1  and  2.2— that  is,  a 
program  fails  if  it  fails  to  meet  all  its  goals— misstatement  of  goals  actuaDy  increases  the  risk 
associated  with  a  program.  In  most  cases,  however,  the  manager  must  accept  sudi  risk  to 
reduce  the  risk  of  losing  overall  support  firr  the  program  to  a  competing  devdofmient 
program.  Managers  well  understand  this  tmsioo  between  the  goals  ofprogram  survival  and 
other  goals  of  the  program;  they  accept  it  essentially  as  a  price  of  entry  for  conducting 
develi^mient  activities.  In  the  end,  however,  sudr  accei^ance  means  that  the  manager 
cannot  expect  to  meet  his  or  her  goals  and  murt  eqiect  to  make  trade-oflb  in  allocating 
shortfidls  among  goals. 
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Whoi  surprises  occur,  the  numager  must  again  make  trade-ofis  anuaig  goals.  Some 
surprises  will  loosen  constraints  on  the  manager;  an  unexpectedly  hi^  performance  outcome 
in  one  area  may  allow  the  manager  to  reduce  risk  associated  with  performance  in  anc^her 
area  or  to  hold  the  line  on  the  costs  or  schedule  of  development.  Negative  surinises,  on  the 
other  hand,  will  lead  a  manager  to  spread  the  negative  effects  across  goals.  A  test  failure,  for 
wrample,  may  lead  to  a  schedule  slip  and  additional  devriopment  woiic  to  achieve  the  initial 
performance  goal  at  the  mcpoise  of  schedule  and  cost  goals. 

How  a  tnawafluw  makes  such  trade-oCfo  should  depend  on  the  rriative  priorities  that  he 
or  she  places  on  difibrent  goals,  based  either  on  guidance  from  hi^er  echelonB  of  government 
overseeing  the  project  or  his  or  her  own  personal  goals.  We  should  expect  these  priorities  to 
differ  from  one  development  program  to  another  and  perhaps  even  to  change  over  the  course 
of  a  development.  Patterns  in  such  trade-offs  are  of  great  interest  to  us. 

Ilthoda  for  Anticipoting  and  Rosponding  to  Risk 

A  manager  can  use  two  basic  approaches  to  plan  for  or  react  to  surprises.  The  first 
em|diaaixes  formal  documents  and  processes.  The  second  approach  emphasizes  good  people. 
The  first  approadi  uses  the  performance  specifications  for  a  new  system  to  set  the  general 
level  of  risk  for  the  program;  more  runbitious  specifications  are  riskier.  The  approach  spells 
out  a  formal  acquisition  plan  for  the  devriopment,  specifying  lines  of  authority,  the  nature  of 
competition,  or  prototyping  used  during  the  development.  It  uses  contracts  and  memoranda 
of  understandiiig  and  agreement  to  balance  the  concerns  of  the  Air  Force  and  other  parties  to 
the  develiqnnent.  It  uses  a  master  test  plan  to  anticipate  required  testing  assets,  set 
sequmioes  events,  and  respond  to  test  foilures  over  the  course  of  the  development.^ 

Broadly  viewed,  a  development  is  a  test  program  that  repeatedly  tests  newly 
developed  sirstems,  analyzes  |»oUenu  identified  during  test,  and  fixes  Ihem  in  preparation 
for  another  test  This  *test-anatyze-fiz,”  or  TAF  approach  applies  at  the  macro  level  as  a 
metaffom  finr  the  program  as  a  whole  and  can  be  ^>idied  in  a  more  targeted,  etytlidt  way  to 
deal  with  specifically  identified  problems. 

Formal  risk  assessment  can  be  associated  with  any  one  rf  these  activities.  Formalrisk 
assessment  works  best  when  Hie  procosaes  in  questitm  are  weD  understood  and  good  data 
exist  on  those  processes.  Hence,  it  is  most  likely  to  support  design  ofselected  parts  of  the 

ssudi  docamenti  create  a  p^wr  trail  that  developere  can  nee  to  cover  themeelvee  if  thinge  go 
wrong;  toat  is,  they  can  prove  that  they  did  eveiythingdiat  was  required  of  them.  Waaremore 
intarMtad  in  fiia  wmy  damloparB  uae  e^  docamente  to  antidpato  risk  and  plan  for  it;  that  is,  we  are 
intarestod  in  how  a  creative  pUammgproceee  can  tailor  doraniente  to  a  develoiHnentprogram*e  needs, 
not  singly  ftdllll  regulatory  requimnents. 


test  program  or  of  warranties  induded  in  a  contract  Risks  associated  with  the  developinait 
as  a  whole,  as  noted  above,  are  more  difficult  to  state  in  dear,  quantitative  terms.  Formal 
risk  assessment  is  (^limited  use  in  such  a  setting.* 

The  presence  of  contracts  among  these  tools  raises  an  important  pmut  about  risk. 

Risk  can  be  perceived  from  different  perspectives.  For  example,  although  a  cmitractOT  is 
probably  better  aide  than  the  Air  Force  to  affect  surprises  that  arise  in  the  day-to-day 
developmoit  of  a  qrstem,  the  Air  Force  is,  presnmaMy,  better  able  dian  the  contractor  to 
nffiBrt-.  overall  funding  for  a  development  program.  Contracts  can  be  written  to  shift  the 
effects  of  surprises  toward  those  parties  best  able  to  mitigate  their  effects.  Hence,  the 
contractor  often  bears  much  of  the  risk  associated  with  unexpected  cost  growUi  during  a 
development,  whereas  the  government  bmus  the  risk  associated  with  premature  termination 
of  a  program.  Mote  generally,  the  Air  Force  as  a  whde  is  pndmbly  better  aide  to  bear  the 
effects  of  surprises  then  ia  an  individual  contractor.  Contracts  can  be  written  to  shift  the 
effects  of  surprises  that  cannot  be  mitigated  to  the  party  better  aide  to  bear  sudi  effects.  In 
practice,  of  course,  even  when  the  Air  Force  as  a  whole  can  bear  large  negative  outcomes, 
officers  in  a  SPO  concerned  about  their  futures  in  the  Air  Force  probaldy  cannot  Hence, 

SPO  managers  may  resist  bearing  rides  best  borne  1^  the  Air  Force. 

Once  a  omtract  is  negotiated,  it  sfdits  the  effects  of  many  surprises  so  that  one  party 
benefits  from  the  surprise  while  the  other  is  hurt  Our  approach  to  risk  implies  that  one 
party  need  not  associate  any  risk  with  a  surprise  that  imposes  a  substantial  risk  on  the  other 
party.  For  example,  under  a  fixed-price  contract,  the  contractor  bears  the  full  risk  <d’ 
unexpected  cost  increases  while  the  Air  Force  feek  no  effect  On  the  other  hand,  the 
contractor  er^ioys  the  full  benefits  of  unexpectedly  low  costs.  When  costs  are  unexpectedly 
low,  the  Air  Force  can  see  such  a  benefit  as  a  foregiHW  benefit  fiir  the  Air  Force— that  is,  as  a 
negative  outcome.  Hut  is,  ahhougfa  the  fixed-price  cmitract  determines  their  costs.  Air  F(»ve 
officialsmay  view  the  finegoae  benefit  as  a  risk  worth  planning  against  Sudi  a  percqdion 
complicates  our  simple  approadi  above  that  risk  is  associated  <ndy  with  n^ative  outcomes. 

Althou^  formal  cmitracta  are  written  only  between  the  Air  Force  and  contractors  or 
between  contractors,  theae  considsratiofu  ^qdy  to  many  other  sitaatioos  in  whidi  more  than 
onepartyplasrsarole.  A  SPO  typically  has  many  relationships  witti  other  SPOs,  test 
fheiUtieo,  other  parts  ofthe  Air  Force,  and  Bomstimes  other  services.  Some  of  these 
relationdiipB  are  codified  in  memoranda  understanding  and  agreement  others  rdy  on 
estabHsfaed  custom.  In  all  cases,  nusre  than  one  perspective  on  tiu  risks  asaociated  with  a 

^For  a  ftntfaar  dtscuBsion  of  these  points,  see  Bodilly,  Camm,  and  Pd,  1981. 
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purtieiilar  sorinise  ia  poMibia  and  can  aflbet  how  managan  plan  tor  and  react  to  that 
surprise. 

The  second  basic  appnadi  to  planning  ftr  and  reacting  to  risks  is  quite  diflEbrent  frina 
that  taken  above.  It  teUea  on  good  petqde  rather  than  documents  and  procedures.  Atsome 
levri,  good  people  are  required  to  negotiate  and  prepare  documents  and  inooedures.  The 
point  here  is  that  a  good  staff  adds  Taluebegmidthoaefunctiooa.  In  foct,  the  primary  value 
of  a  good  staff  may  tsril  lie  not  in  {danning  ftr  the  fiiture  but  in  its  ability  to  react  confidently 
and  ctearively  when  ffunga  go  wnmg. 

The  importance  of  good  peiqile  is  a  point  that  devd<qnna)t  managers  emphasize 
repeatedly.  ContractorB  ftvor  coveted  devehqanant  programs  not  just  with  resources,  but 
with  their  best  people.  The  Air  Force  responds  by  allocating  its  best  people  to  its  hig^iest- 
priority  devalopinent  pngeets.  Sudi  practices  occur  in  productiim  imtgrams  as  wril.  They 
tofc*  on  a  qieeial  m— in  devekqnnent  progranu  because  of  the  nature  of  the  risks 
present  in  those  programs. 

Althflogh  good  pletinitig  can  provide  a  frameworit  for  dealing  with  routine  risks-^isks 
encountered  in  the  past  or  risks  that  are  foiiiy  obvious  in  a  new  program-^t  cannot  manage 
well  the  totally  unexpected.  When  the  totally  nnerqieeted  occurs,  well-informed  and  timely 
discretion  is  required  to  reqKnri  to  the  surprise.  The  better  the  staff  available  to  do  so  is,  the 
better  is  the  response  and  the  less  managers  most  rely  on  the  blunter  rules  that  an 
acquisition  plan  or  contract  miglit  use  to  manage  sorinise.  A  well-organized,  competent  staff 
olfors  an  additional  benefit  in  ffwfisce  of  uncertainty.  Because  surprises  bring  benefits  as 
wdl  as  risks,  the  presence  of  a  solid  staff  allows  managers  to  maintain  greater  flexibility  in  a 
program  to  exphnt  typortunities  as  tb0f  arise. 

The  presence  of  parties  with  dtfforent  points  of  view,  of  course,  complicates  the  use  of 
skilled  petyde  to  respond  flexibly  to  surprises.  Eadi  surprise  offors  opportunities  to  reopen 
an  agroament  made  earlier  to  diange  the  balance  achieved  eariier.  Anumgtheskillsina 
well-ocganiasdstaffwill  typically  be  abilitieB  to  exploit  such  oj^ortonities.  However, 
exptoiting  surprise  to  renegotiate  earlier  agreements  can  damage  the  basic  rdationahip 
betvrsen  two  parties  ever  the  tong  term  ifith^ipeiis  repeatedly,  ultimately  leading  to  mose 
rigid  arrangnmnnts  dosigned  tn  itisnonrap  snrh  mplritstinm.  mm  ifthtij  stifle  thir  flirribility 
that  aUewB  a  program  to  benefit  firampteaaantsarpriaas.  Sodreagdoitatien  is  most  likely  to 
eecnr  whan  die  sidlheftwo  parties  are  not  wdl  balanced  For  exazqde,  if  an  inexperieDced 
8PO  Iheas  a  eontraetar  using  a  team  with  extensive  eqierienee  working  together,  we  can 
aotyoct  trouble  downstream  as  surprises  provide  <g>portonities  for  the  contractor  to  explmt  its 
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SUHMARY 

Ahhooi^  our  i»rimaiy  interest  lies  in  risk  assessment  and  risk  managemoit  in 
devetopment  programs,  the  nature  of  development  activities  suggests  that  we  should  be 
prqtared  to  wramine  a  fairly  laead  range  of  management  activities.  Hus  is  true  despite  our 
narrow  definition  crfrisk  as  the  presence  of  unpredictahle  events  with  negative  consequences. 
Idany  development  managers  view  risk  this  way  and  see  their  task  as  general  managers  as 
one  of  idortiQring  and  controlling  such  risk  over  file  course  of  a  devdii^MBent. 

Surprises  can  arise  outside  a  develoinnent  program — beyond  the  control  of  those 
invtdved  in  the  development— or  much  doeer  to  home  and  doeer  to  their  contrdL  Thi^can 
aflbct  the  basic  survival  of  a  inegrun  or,  assuming  that  it  survives,  the  cost,  sdiedule,  and 
performance  associated  with  the  program.  Devdofunent  managers  use  two  diffment 
ai^liroachee  to  plan  for  and  react  to  surprises.  develop  documents  and  procedures  that 
define  both  risk  and  the  methods  fin  reducing  or  redistributing  il  These  indude  system 
specificati<ms,  acquisition  plans,  contracts,  memoranda  of  understanding  and  agreement,  and 
test  plans.  And  devekqunent  managers  develop  and  nurture  esperienced,  skilled  stafib. 
Without  such  staflb,  managers  caimot  respond  adequately  to  the  wide  range  of  surinises  that 
arise,  ^l^th  them,  they  can  rdy  less  on  formal  documents  and  processes  and  thereby 
maintain  the  flexibility  that  allows  tlmn  to  take  advantage  of  new  opportunities  as  th^ 
arise. 

In  the  end,  then,  it  is  difficult  to  distinguish  risk  management  finom  general 
management  In  the  fidlowing  sectums,  we  explore  a  broad  range  of  managemmit  issues  to 
understand  better  how  managers  have  assessed  and  managed  risk  in  the  F-16  MSIP. 
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3.  THE  F-16  SYSTEM  PROGRAM  OFFICE  AND  tTSIIANAOEMENT 


By  Die  time  MSB*  befen.  Die  F-16  SPO  wee  e  laife,  wdl-organised  ecDvity.  TIm  F-16 
progrem  began  in  1971.  Aa  Dm  SPO  and  General  Dynanuce  began  to  idan  MSB*  in  1979, 
Di«y  wore  eompleDng  lull  ecale  davdofanait  of  the  F>16A/B,  freeing  management  reaoiiroes 
finr  further  deveiepaMntworiL  Hm  management  taehnuiuea  and  eqwrienoe  developed 
daring  the  1970b  inevided  a  good  beau  fior  a  continuing  devdf^Mnent  program.  Thiaaection 
deaetibea  the  SPO  that  managed  MSB’dnriiv  the  1960a.  It  briefly  reviewa  the  changes  in 
that  organiaation  over  time  and  how  that  diange  related  to  MSB*  itaelf.  It  alao  examines  the 
continuity  of  leadership  moat  rdevant  to  MSIP. 

ttSTlAL  ORQANBAT10N 

TaUe  3.1  shows  the  basic  organisation  fitr  the  SPO  when  planning  began  for  MSB*. 
The  SPO  used  a  matrix  otganixation  with  three  project  offices  for  production — for  the  U.S. 


Tables.! 

Ocgaafsation  of  Dm  P>1S  SPO  Daring  MSB* 


Organiaation  in  1979 

Oiganiution  in  1988 

Office 

Assigned 

Assigned 

Sjrmbri 

Directorate  Name 

Persannel 

Directorate  Name 

Personnel 

YP 

Deputy  fur  F-16 

36 

Deputy  ftr  F-16 

9 

Prefect  Directorates 

YPD 

Deployment  and  test 

28 

Development  and  integration 

28 

YPF 

Test  ai^  deidoynient 

22 

YPR 

ATiQMII 

23 

TPX 

MnItiMtinMl  pwngf  im 

21 

Multinational  programs 

36 

Functional  DiiectoratM 


YPA 

Acquiaitian  logistics 

46 

YPC 

Configuration  management 

36 

Configuration  management 

28 

YFB 

Fngineering 

66 

Engineering 

102 

TPK 

PWIW,— tMl* 

41 

Contracting 

66 

ypL 

Loaistios  alaimina 

61 

Logistics 

60 

TPM 

27 

ManufiMturingGA 

18 

TPO 

19 

Mawgam^nit  <i|mf 

13 

YPP 

Progrw  contrel 

66 

i^ro^TUQt  ooptrol 

40 

TPS 

^rstemsafrty 

2 

80UBCS:  UJ.  Air  Fitw,iUrQnauHeriOjrileiiaPWriio.  P-16  DireetorUrflinagaiiMitOpwotieni. 
‘SaaiaaBBal  Air  Flaw  BiM>,OhiOtJaiiuMT-Jun»  1878  and  July-Dwiaber 
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Air  Force  (YPD),  for  foreign  ealee  (YPX).  and  6>r  new  development  programs  (YPR)— and 
fimctional  offices  typically  found  in  a  SPO.  It  was  commanded  by  a  migor  general.  TImSPO 
was  assigned  170  people  and  had  an  additional  281  collocated,  for  a  t(^  of  451.  Of  these, 
178  were  officers,  22  were  airmen,  and  251  were  civilians. 

Primary  planning  for  the  MSIP  occurred  in  the  durectorate  for  prcgects,  YPR,  which 
had  responsibility  for  integrating  new  sub^stems  with  the  P-16  as  they  became  available. 
Before  MSIP  existed,  this  directorate  planned  integration  subqntmn  by  subqrstem.  For 

individual  efibrts  were  under  way  cm  the  APG-68,  AMRAAM,  LANTIRN,  head-up 
display,  GPS,  and  many  other  subsystems  that  would  subsequently  be  associated  with  MSIP. 
With  the  advent  of  MSIP,  this  directorate  continued  to  manage  subsystem  integrations 
individually,  but  MSIP  allowed  the  directorate  to  do  so  within  a  broader  framework.  As 
MSIP  became  active,  YPR  managed  it  as  a  separable  entity,  with  its  own  goals  and 
milestones.  By  1981,  this  managmnent  task  manifested  itself  organisationally  in  a  growth- 
management  group  (YPR-1)  within  YPR  that  was  responsible  only  for  MSIP. 

Outing  the  formative  stages  of  MSIP,  the  directorate  for  projects  was  run  by  a 
lieutenant  odimel  and  dominated  by  mihtaty  perstmnel— 18  officers  to  six  civilians,  four  of 
whom  were  seoretaries. 

Offices  on  the  functional  side  of  the  matrix  supported  the  directorate  for  projects. 
Those  offices  were  typically  directed  by  officers  but  dominated  by  civilians.  The  directorate 
for  engineering  played  a  special  role  for  the  snbqrstems  that  became  associated  with  MSIP: 
Directed  by  a  civilian,  it  provided  the  in-house  technical  expertise  required  to  oversee  the 
integration  and  production  inonporation  (d'thooe  subsystems. 

ORQAMZATKNIAL  CHANGES  DURMG  THE  19e0t 

During  the  1980s,  the  SPO  underwent  a  number  of  cdianges.  TaUe  3.2  provides  a 
quick  overview  of  this  period,  showing  number  of  personnd  assigned  to  the  SPO  as  a  whole 
and  to  three  directorates  associated  with  IfSIP.  MSIP  accounted  for  a  significantiy  hi^er 
pnqiostum  of  those  directorates*  management  interest  during  the  1980s  than  in  other 
directoratea.*  The  table  shows  that  total  staffing  dedined  gradually  until  1981,  when  it 
staUHaad  and  began  a  gradual  rise.  The  increase  continued  into  the  mid-1980s.  Budgetcuts 
beginning  in  1987  required  significant  cuts  that,  as  we  shall  see,  affected  SPO  management 
MSIP  activities. 


iTMs  jodgaisnt  is  based  on  a  review  of  dimetorate  historical  reports  from  the  period. 
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TablaSJI 

AasIgiMd  PerMHUid  in  the  F-16  8PO 


Six-Month  Date 

Total 

Directorate  Symbol 

YPD 

YPE 

YPR 

July  1979 

461 

28 

65 

23« 

January  1980 

442 

28 

71 

22* 

June  1980 

434 

26 

69 

lO* 

January  1981 

miMring  — 

June  1981 

399 

27 

74 

19* 

Deoemher  1981 

401 

31 

72 

20* 

June  1982 

425 

37 

76 

21* 

Deeemher  1982 

437 

36 

78 

25* 

June  1983 

419 

34* 

80 

17* 

Deoemher  1983 

445 

41* 

86 

21* 

June  1984 

—  miMing  ^ 

December  1984 

466 

33* 

111 

21» 

June  1985 

473 

38* 

106 

20* 

December  1986 

480 

37* 

111 

23» 

June  1986 

472 

40* 

112 

20* 

December  1986 

478 

41* 

111 

23» 

June  1987 

440 

36* 

109 

0 

December  1987 

414 

33* 

102 

0 

June  1988 

417 

31« 

104 

0 

December  1988 

404 

28* 

102 

0 

SOUBCE:  U^.  Air  Force,  Aeronautieal  Systems  Division,  F-16  SPO  Directorate  of 
lianegement  Opemtioiis,  “Semisnimsl  Historical  Beport,*  Wr^t>Patterson  Air  Force 
Base,  various  dates. 


*MSIP  activities  occurred  primarily  in  these  prcgect  offices. 

During  this  period,  the  first  nuyor  MSIP-ielated  oiganizational  change  occurred,  in 
1983.  The  SPO  reorganized  to  transfer  responsibility  for  MSIP  firom  the  directorate  for 
projects  to  a  new  directorate  built  around  the  old  directorate  of  deplc^yment  and  test: 


As  the  QdSlP}  prognun  moved  out  of  the  realm  of  a  future  program  and  more  into  the 
area  of  a  production  aircraft,  the  dedaion  waa  made  to  move  the  program  into  a  division 
more  adept  at  handling  integratioa,  testing,  and  deployment* 


This  new  directorate,  which  would  become  known  as  the  directorate  of  integration  and  test, 
efifoctively  became  the  inroject  ofiBce  for  the  F-16G/D.  A  new  directorate  of  field  operations 
(YPF)  took  on  responsibility  fiir  the  F>16A/B. 

The  old  directnrate  of  progects,  now  the  directorate  of  development  plans,  retained 
responsibility  for  selected  subsystems  relevant  to  MSIP,  including  the  Alternate  Fii^ter 
Engine  and  GPS.  It  oontinued  to  oveiaee  new  techmdogies  that  mi^t  be  integrated  into  the 

>UA.  Air  Force,  Aenmantical  Systems  Division,  F*16  SPO  Directorate  for  Development  Flans, 
"Semiannual  IBstorical  Report,  1  Januaiy-SO  June  19^,*  Wrii^Patterson  Ah  Force  Bm,  Ohio. 
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F-16  throu^  MSIP  in  future  configurations.  And  it  continued  to  oversee  the  develoinnent  of 

« 

variations  on  the  F-16  other  than  the  F-16C/D.  Meanwhile,  the  directorate  of  integration 
and  tMt  took  over  respondbility  for  subiQrsteins  fimnally  configured  as  part  of 
forthcoming  blocks  of  the  F-16C/D,  including  the  APG-68,  AMRAAM,  LANTIRN,  HUD,  and 
many  other  subsystems.  About  half  of  the  staff  finm  the  old  directmate  of  projects  moved  to 
the  new  directorate  of  int^rati<m  and  test  during  this  reorganization,  ensuring  a  fair  degree 
of  continuity  despite  the  change.  Nonetheless,  the  diange  meant  that  subssrstems  relevant  to 
MSIP  would  be  managed  bF  two  separate  project  offices  within  the  F-16  SPO  for  the  next 
several  years. 

The  directorate  of  integration  and  test  continued  to  be  led  by  the  original  director,  a 
colonel.  Officers  exceeded  civilians,  22  to  11.  A  lieutenant  colonel  continued  to  lead  the 
directorate  of  development  plans,  where  officers  continued  to  exceed  civilians,  14  to  three. 

The  second  mi^r  MSIP-related  change  occurred  in  1987,  when  the  directorates  of 
integration  and  test  CYPD)  and  devdopment  plans  (YPR)  merged  to  become  the  directorate  of 
development  and  int^ration  (YPD).  Utilike  the  first  change,  this  change  resulted  firom  a 
budget  cut  that  forced  the  F-16  to  reduce  its  staffing  and  consolidate  activities.  But  like  the 
first  change,  this  one  maintained  continuity  by  transferring  personnel  with  their  associated 
tasks  from  one  office  to  another.  The  old  YPR  simply  disappeared.  Test  activities  in  the 
directorate  of  integration  and  test  migrated  to  the  directorate  of  field  operations,  leaving  the 
new  YPD  essentially  as  a  program  management  directorate,  overseeing  the  F-16C/D 
program,  integration  of  govemment-fumished  subqrstems  associated  with  MSIP,  and 
development  of  new  derivative  engines.  MSIP-related  staffing  in  the  project  offices  decreased 
proportionately  more  than  staffing  in  the  SPO  as  a  whole,  requiring  the  managers 
responsible  for  MSIP  activities  to  relinquish  significant  responsibilities  to  the  functioiud 
directorates  on  the  other  side  of  the  matiiz.  Since  1987,  aU  MSIP-related  activities  have 
been  managed  throu^  the  YPD  office,  with  the  continuing  support  of  the  functional 
directorates. 

TaUe  3.1  displays  the  structure  of  the  SPO  at  an  important  milestone  near  the  end  of 
the  19808:  General  Dynamics’  delivery  of  the  first  MSIP  m.  Block  40  F-16C,  in  December 
1988.  When  that  ooenned,  the  F-16  SPO  had  404  personnel  assigned,  induding  132  officers 
and  258  civilians.  Despite  the  predominance  of  civilians,  officers  ran  all  but  two  directorates. 
One  ofthe  two  was  the  directorate  ofengineering,  where  civilians  exceeded  officers,  79  to  23. 
Officers  continued  to  predominate  the  project  directorate  responsiUe  for  MSIP,  the 
directorate  of  development  and  integratiim,  21  to  seven.  The  SPO  continued  to  be 
commanded  by  a  mgjor  general. 
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CONTmurTY  OF  PERSONNEL 

Short  tours  of  duty  tend  to  h«mp«w  continuity  over  the  course  of  a  long  development 
program.  The  predominance  of  military  personnel  in  leaitership  positions  and  in  the  project 
directorates  with  greatest  responsibility  for  MSIP  raises  questions  about  continuity  in  the 
SPO.  Table  3.3  sununarizes  information  on  the  principal  managers  relevant  to  MSIP. 

Althoui^  the  taUe  is  not  complete,  the  data  available  tell  a  fairly  dear  story.  Recall 
from  Table  3.2  that  YPO  did  not  actually  become  rdevant  to  MSIP  until  1983  and  that  YPR 
disappeared  after  1986.  The  SPG  commanders  appear  to  have  served  standard  three-year 
tours.  Their  deputies  were  also  military,  limiting  the  institutional  memory  developed  at  the 
top  of  the  organizatioiL  Perhaps  the  most  important  source  of  continuity  at  the  top  of  the 
organization  has  been  Mr.  John  Brailoy,  the  technical  director  for  the  SPO  since  1983  and 
the  director  of  engineering  for  a  short  time  before  that 

Military  managers  have  also  run  YPD  and  YPR,  where,  for  the  most  part,  they  appear 
to  have  served  two-  to  three-year  tours.  Their  deputies  and  most  of  their  professional  stafis 
have  also  been  military,  suggesting  that  any  institutional  memory  about  activities  in  these 


Table  3.3 

BCanagen  Reievant  to  MSIP 


Year 

Commanders 

F-16SPO 

Directors 

YPD 

YPE 

YPR 

1979 

Abtahamson 

Belinne 

Bair 

Paddn 

1980 

Wolff 

Bair 

? 

Monahan 

1981 

Monahan 

Wolff 

Madden? 

? 

1982 

Monahan 

WolffSabo 

BraOey 

Boyd/ 

Westover 

1983 

Monahan/Yates 

Sabo 

Brailey/Culp? 

Westover 

1984 

Yates 

Sabo 

WestovenTucker 

1965 

Yatea 

Sabo/Hayaahi 

Tucker/Cathey 

1966 

Yataa/Eai^ 

Hayashi 

LeBiaster 

Cathey 

1967 

Ea^et 

Hayaafai/Hogstmn 

tjtMamtar 

1966 

Ea^et 

Hogstnna 

LaMaster/ 

Smithers 

SOUBCB8:  1T.S  A<>  gww,  AjMMntiMl  Dhriainn,  nrtwl  P.1S  oty niMtiwi  Wright. 

Pattenon  Air  Fom  Baw,  (MliD,  and  SAfPAR  biMiiigt  from  Um  period  ahown. 


areas  would  lie  in  the  functional  directorates  that  supported  those  activities.  The  directorate 
of  engineering,  YPE,  might  be  a  Ic^cal  place  to  turn  for  such  knowledge  about  the  tedinical 
aspects  of  the  development 

This  final  directorate,  YPE,  has  usually  been  run  by  a  civilian  with  a  military  deputy 
and  has  used  a  predominantly  civilian  staff.  The  directorate  of  engineering  achieved 
considerable  stability  in  management  from  1984  on.  For  several  years  before  1984,  during 
the  formative  years  of  MSIP,  leadership  of  this  directorate  experienced  much  more  rapid 
turnover. 

As  a  working  hypothesis,  we  might  postulate  that  institutional  memory  in  the  SPO  is 
limit/wl  The  SPO’s  leadership  has  stabilized  in  recent  years,  but  we  would  have  t  seek 
stability  in  the  early  years  of  MSIP  farther  down  in  the  civilian  parts  of  the  organization, 
presumably  on  the  functional  side  of  the  matrix. 

SUMMARY 

MSIP  is  only  one  activity  among  many  in  the  mature  F-16  SPO.  In  the  face  of 
dramatic  ffhangaa  in  MSIP,  the  SPO  as  a  whole  has  fluctuated  in  size  only  about  20  percent, 
sometimes  in  response  to  MSIP-related  changes,  more  often  in  response  to  totally  unrelated 
factors.  The  SPO  was  already  well  organized  and  experienced  as  an  organization  ly  the  time 
MSIP  began  in  1979.  The  predominance  of  the  military  in  the  SPO’s  leadership  and  in  its 
project  offices  probably  hampered  accumulation  of  knowledge  about  the  system  at  a  hi^ 
level.  But  the  SPO  appears  to  have  accommodated  MSIP  comfortably  as  a  new  activity  as 
MSIP  matured. 

MSIP  began  as  a  development  concept  in  the  part  of  the  SPO  devoted  to  such  work, 
the  directorate  of  projects.  As  MSIP  matured  and  approached  the  point  of  being  embodied  in 
a  msyor  new  F-16  variation,  the  F-16C/D  aircraft,  the  SPO  reorganized  to  accommodate  that 
change.  In  1983,  a  new  project  directorate  was  effectively  set  up  to  house  the  new  F-16C/D. 
As  portions  of  MSIP  matured  enou^  to  be  incorporated  in  production,  ffiey  came  to  be 
managed  in  the  new  directorate.  MSIP-related  subtystems  at  a  more  devdopmental  stage 
remained  in  the  directorate  of  developmental  plans.  This  approach  presents  MSIP  more  as  a 
concept  or  plan  than  as  a  formal  program;  it  allowed  the  SPO  to  handle  individual  activities 
associated  with  MSIP  much  as  it  would  have  handled  other,  similar  activities  unrelated  to 
MSIP. 

Large  budget  cuts  in  1987  forced  an  end  to  this  approach.  To  accommodate  reductions 
in  staffing,  the  SPO  reorganized  again  and  placed  all  MSIP-related  worir  in  one  directorate. 
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where  it  remains  today.  The  SPO  also  significantly  increased  its  reliance  on  its  dvilian- 
dominated  fiinctional  directorates  as  a  result  of  the  change. 

In  sum,  MSIP  is  one  activily  among  many  in  the  F-16  SPO.  It  has  been  important 
enough  to  change  the  SPO  as  a  whole.  But  it  was  conceived  in  and  continues  to  operate  in 
the  broader  context  of  the  F-16  program  as  a  whole. 
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4.  F-16  DEVELOPIIENT  PROQRAIIS  DURING  THE  ISBOs 


Even  after  the  doae  of  the  initial  full-scale-development  program  for  the  P-16  in  1979, 
the  F-16  SPO  maintained  an  active  researdi-and-development  proeram.  Table  4.1  presmats 
the  program’s  spending  <m  activities  rdated  directly  to  reseaida  and  development  Although 
the  bulk  of  real  spending  occurred  during  the  1970s,  a  substantial  effort  has  continued  to  the 
present 

MSIP  accounts  for  part  that  effort  But  MSIP  is  only  one  of  a  family  of  related 
programs  to  upgrade  new  production  F-16s,  retrofit  new  capabilities  into  existing  F-16s,  and 
develop  new  variations  based  on  the  F-16.  These  programs  use  research-and-development 
funds,  but  they  rely  primarily  on  production  funds  to  pay  for  the  nonrecurring  costs 
associated  with  production.  Hence,  the  figures  in  Table  4.1  offer  only  a  lower  bound  on 
development  spending,  because  all  these  programs  share  the  common  characteristic  that 

Table  4a 

BOTJkE  Expenditures  in  the  F-16  Program 


FiacaJ 

Year 

Ezpandituraa  ($  millions) 

Iban-Y ear  Dollars 

1980  Dollars 

1975 

32.0 

48.0 

1976 

214.7 

290.2 

1977 

69.0 

88.5 

1977T 

256.4 

322.0 

1978 

162a 

192.5 

1979 

93.6 

102.4 

1980 

27.6 

27.6 

1981 

43.1 

38.5 

1982 

613 

47.4 

1983 

70S 

663 

1984 

93.1 

70.0 

1985 

90.6 

65.8 

1986 

61.1 

433 

1987 

52.0 

35.8 

1988 

21.7 

US 

1989 

2AA 

15.6 

1990 

18.0 

11.1 
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PiTirioB,  P.M  gPO,  F-U  Stiteltd  AequiMon  Etpert. 
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in  tin  seaw  docBBrat. 

T  ■  TnuitiaB  qneiter  (diily-SapUmiMr  1977). 
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they  upgrade  the  F>16  primazily  adding  new  capabilities  essentially  off  the  shelf  firom 
development  programs  conducted  elsewhere.  MSIP  development  work  concerns  itself 
primarily  with  integrating  the  new  ^retems  with  a  baseline  version  of  the  F-16  and  ensuring 
that  all  systems  integrated  at  the  same  time  are  compatible  with  one  another.  Seeing  MSIP 
in  the  context  of  a  fBunily  of  gmilar  programs  reminds  us  that  the  development  approach 
provided  1^  MSIP  is  as  mudx  a  way  of  life  in  the  F>16  SPO  as  it  is  a  formal  program  aimed 
only  at  bringing  the  F-16(VD  on-line. 

This  section  briefly  reviews  a  series  of  programs  from  the  1980s  that  created  new 
aircraft  derived  from  the  F-16,  upgraded  the  F-16A/B,  and  sought  to  provide  an  aircraft 
rfftmgn  that,  could  meet  the  Army’s  needs  for  a  new  close-air-support  aircraft.  Table  4.2 
capsulizes  relevant  program  activities.^ 


TUble4J2 

F-16  Oevetopment  Programs  During  the  1980s 


Name 

Date  Begun 

Brief  Summary 

New  Variations  on  the  F-16  Aircraft 

Agile  Falcon 

1988 

Would  have  enlarged  the  wing  and  improved 
aerodynamics,  avionics,  and  other  internal  systems. 
Canceled  by  the  secretary  of  defense. 

RF-16  Reconnaissance  Aircraft 

1986 

Would  add  a  sensor  pod  and  software  to  support 
reconnaissance  functions. 

F-16N  Navy  Adversary  Training 
Aircraft 

1987 

Emulates  Soviet  filter  characteristics;  Navy  bou^t 
26  and  considered  more. 

FSX  Joint  Development  IVogram 
with  Japan 

1987 

F-16  provides  the  baseline  for  nu^r  Japanese 
improvements  in  airframe,  avionics,  weaponry,  and 
radar  cross  section. 

Operational  Upgrades  for  the  F-16A/B 

First  Operational  Upgrade 

1987 

Added  selected  F-16C/D  capabilities  to  Blocks  10  aiul 
16  F-16A/Bb  sold  to  EPG  air  forces. 

F-16ADF  Air  Defense  Fighter 

1986 

Modified  270  F-16As  for  use  in  the  Air  National 

Guard. 

Mid-life  Upgrade 

1989 

Would  upgrade  existing  U.S.  and  EPG  F-16A/Bs  to 
have  capalnlities  similar  to  F-16C/Ds. 

iThe  discussions  bdow  are  based  on  interviewe  as  wdl  as  General  Dynamics,  1990;  Richardson, 
1990;  and  W<df,  January  1986-Deoeniber  1988.  Our  description  also  relies  on  official  Air  Force 
histories  of  various  Aeiooautieal  Systems  Division  development  programs  written  by  C .  J.  Geiger 
during  October  1989-Deoember  18M.  Because  that  diseu^on  is  ftro  an  undassified  hiatoiy  in  a 
rlaeaiflad  Air  Force  document,  we  cannot  provide  an  explicit  citation  in  this  document 
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NEW  VARIATIONS  ON  F-16  AIRCRAFT 
Agte  Falcon,  SucoMsor  to  the  F*1SC/D 

The  new  variation  most  doaely  related  to  MSIP  was  Agile  Falcon,  which  General 
Dynamics  marketed  as  MSIP  IV,  a  designation  that  could  be  viewed  as  an  attempt  to 
ininifniitA  the  degree  of  change  in  moving  from  the  F-16C/D  to  a  new  aircraft  designed  to 
replace  it.  It  is  also  an  indication  of  the  extent  to  which  MSIP  has  become  a  way  of  doing 
business  in  the  F-16  SPO.  The  managemmit  techniques  used  in  the  original  three  stages  of 
MSIP  that  yielded  the  F-16C/D  natuiaUy  carried  over  into  the  design  of  a  successor  aircraft. 

Agile  Falcon  was  the  product  of  a  three-year  design  effort  at  Cieneral  I>3mamic8.  It 
provided  the  Hamb  for  a  response  to  a  1987  request  from  the  secretary  of  defense  to  develop 
an  upgraded  version  of  the  F-16  that  could  replace  the  F-16C/D  and  complement  the 
advanced  tactical  fighter  that  would  replace  the  F-15.  The  new  design  offered  a  larger  wing 
and  amodynamic  improvements  in  addition  to  the  changes  in  avionics  and  other  systems 
internal  to  the  airframe  that  characterized  the  first  three  stages  of  MSIP. 

In  1988,  the  deputy  secretary  of  defense  approved  a  two-year  pre-development 
program,  with  full-scale  devel<q»nent  to  begin  in  FY  90,  pending  approval  by  the  Defense 
Acquisition  Board,  and  production  deliveries  to  the  Air  Force  to  begin  around  1995.  With  the 
deputy  secretary’s  support,  the  F-16  SPO  reached  an  agreement  with  the  advanced  tactical 
fighter  SPO  to  transfer  engine  and  avionics  technology  from  the  new  filter  for  use  in  the 
design  of  AgQe  Falcon.  Institutional  arrangements  were  also  established  with  the  European 
Participating  Governments  to  facilitate  their  participation  in  the  multinational  F-16 
program.  The  new  secretary  of  defense  canceled  the  program  in  1989. 

This  variation  adds  a  sensor  pod  with  multiple  capabilities  to  a  standard  F-ldC/D  in 
much  the  same  way  that  MSIP  added  individual  subqrstems.  As  a  point  of  reference,  the 
Dutch  have  been  using  a  modified  F-16A  since  the  early  19808  as  a  platform  for  a  variety  of 
European  sensor  pods. 

The  RF-16  variation  retains  all  the  air-to-air  and  air-to-ground  combat  capabilities  of  a 
standard  F-lfiCVD.  Like  LANTERN,  the  sensor  pod  has  a  modular  design  optimized  tor  F-16 
aerodynamics,  maintainability,  and  fit  Even  with  the  pod  in  {dace,  the  aircraft  retains  its 
full  flight  onvelope.  The  variation  affects  primarily  pod  integration  and  control  and  an 
expansion  of  F-16  software  to  suiqtort  reconnaissance  functions.  The  pod  is  int^prated  with 
the  P-16  HUD  and  other  diqtlqys  and  with  cockpit  controls  that  allow  man-in-the-loop  sensor 
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control  to  imiHOve  target  acquisition  and  enable  in-fli^t  changes  in  sensor  missions.  The 
ssrstem  supports  real-time  viewing  in  the  cockpit  and  near-real-time  viewing  on  the  ground. 

General  Ilynamics  developed  the  pod,  simidifying  the  integration  process.  Concept 
validation  occurred  in  1986.  Flight  simulations  were  used  to  verify  the  cockpit  arrangements. 
The  U.S.  Air  Force  selected  the  RF-16  as  a  successor  for  the  RF-4C  in  1989.  The  aircraft 
entered  the  DoD  ^>proval  cyde  in  1990,  with  full-scale  devdopment  expected  to  begin  in 
FY92.  ICSn*  has  not  required  formal  OSD  approval.  But  the  general  approach  to  conceiving 
and  developing  the  RF-16  is  wwiilar  to  activities  for  a  block  change  in  MSIP. 

F-16N  Navy  Adversary  Training  Aircraft 

General  Dynamics  developed  and  produced  this  aircraft  for  the  Navy,  using  an 
approach  to  that  for  developing  Air  Force  variations  on  the  F-16.  The  Navy  sou^t 

an  aircraft  that  could  emulate  fourth-generation  Soviet  filter  perfmrmance,  qmtem 
«>apyhilitiaa,  anH  tartira  It  modified  a  Block  30  F-16C/D  to  provide  those  characteristics.  For 
the  most  part,  it  simply  selected  from  the  same  set  of  subqnstems  for  developing  Air  Force 
variations.  The  Navy  bought  22  sin^e-seat  and  four  two-seat  versions  in  1987-88  and  has 
considered  additional  purchases  of  the  same  design.  This  devdlopment  illustrates  the 
feasibilify  of  using  an  BfSIP-like  approadi  for  a  very  small  block  size. 

FSX  «loint  Devetopment  Prognm  with  Japan 

The  F-16  aiiftame  provides  the  starting  point  for  a  new  Japanese-U.S.  filter  that  will 
replace  the  Japan  Air  Self-Defense  Force  F-1,  developed  by  Mitsubishi  Heavy  Industries. 

The  design  and  development  processes  for  this  derivative  differ  substantially  firom  those  for 
MSIP.  Althougdt  both  programs  are  multinational,  the  Japanese  will  clearly  dominate  the 
FSX  development,  just  as  the  U.S.  dominates  the  F-16  MSIP.  Mitsubishi  Heavy  Industries 
will  be  the  prime  contractor;  General  Dynamics  will  serve  as  a  subcontractor,  along  with 
many  other  U.S.  and  Japanese  firms.  The  U.S.  Air  Faroe  SPO’s  role  will  be  cmnmensurately 
limited.  And  the  FSX  will  look  quite  different  from  the  F-16  on  which  it  is  based:  Only 
20-30  percent  of  the  original  airframe  will  remain  unchanged,  and  the  design  will 
incorporate  Japanese  radar-absorbing  materials  to  reduce  the  radar  cross-sectitm,  primarily 
Japanese  avumics,  and  J^mnese  weapmny. 

The  Japanese  selected  the  F-16  as  the  basis  for  its  FSX  in  1987  and  completed  a 
memorandam  of  understanding  to  that  eCfoct  a  year  later.  The  agreement  stared  great 
pditical  controversy.  The  U.S.  government  finally  agreed  to  the  arrangement  in  1989, 
allowing  a  jdnt  design  team  to  b^inworic  in  1990.  First  flight  of  a  prototype  is  expected  in 
1998,  with  production  to  occur  around  the  tom  of  the  century. 
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In  sum,  MSIP  does  not  offer  the  only  model  availaUe  for  develoinng  F>16  derivatives. 
This  hi^visibility  ahemative,  effeetivdy  cmitrolled  a  foreign  government  and  prime 

oontraetor,  and  relying  primarily  on  subsystems  nt^  traditionally  associated  with  the  F-ie 
program,  represents  a  veiy  different  modd  of  multinational  development 

THE  OPERATIONAL  CAPABUXTY  UPGRADE  PROGRAM  FOR  THE  F-ISAm 

From  the  beginning,  MSIP  was  conceived  as  a  program  that  would  be  implemented  in 
part  by  retanfitting  capabilities  into  a«i«ting  aircraft  as  new  capabilities  became  available. 
Althoui^  sudi  retrofits  were  carefully  planned  into  the  designs  of  new  F-IGC/Ds,  many  could 
also  prove  useful  in  the  F-16A/B.  In  fact,  as  the  U.S.  Air  Force  moved  toward  the  F-16GD 
model,  foreign  sales  of  the  F-16A/B  were  threatened  by  a  concern  that  the  U.S.  would  lose 
interest  in  and  provide  less  suiqxnt  for  the  less-capable  F-16A/B  model.  One  way  to  avert 
such  concerns  was  to  upgrade  F-lfiA/Bs  using  capaldities  brou^t  to  the  F*16  program 
throui^  MSIP.  The  F*16  pit^ram  developed  the  Operational  Capability  Upgrade  (OCU) 
program  to  do  just  that  Like  MSIP,  it  proceeded  under  limited  oversi^t  from  DoD. 

MSIP  provided  two  important  resources  for  the  OCU:  a  database  of  available 
capabilities  that  could  be  considered  for  incorporation  in  various  versions  of  the  F-16A/B,  and 
management  techniques  and  capabilities  in  the  F-16  SPO  and  General  Dynamics  that 
facilitated  such  upgrades. 

Upgrades  provided  throu^  the  OCU  program  produced  three  separate  versions  of  the 
F-16A/B. 


First  Operational  Capability  Upgrade 

In  1987,  the  first  OCU  changes  affected  existing  Block  10  and  Blodr  15  F-16A/Bs  and 
new  Block  IS  F>16A/Bs  sold  to  the  European  Participating  Governments.  Much  like  those  in 
a  block  change  MSIP,  the  changes  indude  expansion  of  computer  capadty;  provisions  for 
beyond-visual-range  missiles;  and  additiou  ai  a  radar  altimeter,  the  wide-an^e  HUD  used 
in  the  F*16Cff),  and  the  new  FlOO-PW-220  Alternate  Fi^^ter  Engine.  BdSIP  had  previously 
integrated  each  of  these  modifications  into  F>16  variants,  simplifying  the  taek  of 
impfementmgthem  on  F-16A/B  aircraft. 

The  F-lfiADF  Air  Defense  FigMer 

The  F>16ADF,  or  F>16A  (ADF),  was  a  respmise  to  a  1986  U.S.  Air  Force  proposal  to 
develop  a  new  aircraft  to  replaoe  F>48  and  F-106s  as  air  defense  interceptors  in  the  Air 
National  Guard.  The  Air  Force  fevmed  an  aircraft  designed  as  a  modification  of  an  ««i«Hng 
fif^hter,  but  considered  a  range  of  alternatives.  Ihe  Aeronautical  $3ntems  Division  (ASD),  in 
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coordination  with  the  Tactical  Air  Command  and  Headquarters,  U.S.  Air  Force,  developed 
aoquiattico  and  sonroe-aelection  plans  that  ultimatdy  unaided  a  (todsion  later  in  the  year  in 
&vtnr  cf  modifying  the  F-IAA.  The  primary  cunpetitor  was  the  Northrop  F-20.  Afierthe 
award  of  a  contract  to  modify  270  F-l€Aa  already  in  the  inventory,  ASD  transferred  control 
of  the  program  to  the  F-16  SPO.  Production  deliveries  (d*  modified  aircraft  were  scheduled  for 
1989-91. 

This  derivative  develoianent  program  began  very  differently  than  the  first  OCU 
program  (or  MSIP)  fay  using  a  competition  to  discipline  the  designers.  General  Dynamics 
was  able  to  respond  quickly  to  the  Air  Force’s  expressed  interest  because  the  (X3U  program 
was  in  place.  The  F-16ADF  simply  offered  an  additional  apphcatiim  of  the  system  that 
yielded  the  first  OCU  changes  described  above.  Hence,  General  Dynamics  was  already 
familiar  with  the  subsystem  integrations  that  it  proposed  for  transforming  an  F-16A  into  an 
F-16ADF.  For  example,  the  F-lfiAOF  upgraite  would  modify  the  APG-66  fire-control  radar  to 
accept  AMRAAM  data.  It  improved  the  F-16A’s  electronic  counter-countermeasures,  hi^- 
frequency  radio,  identification,  firiend-or-foe  (IFF)  system,  and  fli^t  data  recorder,  and 
added  &oup  A  provisions  for  GPS.  MSIP  had  included  each  of  these  improvements  in 
earlier  F-16  variants. 

MM-Ula  Update  Program  for  tha  F-16A/B 

This  program  is  essentially  to  retrofit  existing  F-16A/B  aircraft  in  the  U.S.  and  the 
European  Participating  Governments  so  that  their  avionics  are  very  dose  to  those  in  an 
F-16CyD.  To  allow  transfer  of  CCTtain  technologies,  this  program  has  required  a  change  in 
the  secuiify  agreements  in  place  fin  the  governments  involved.  Ihe  F-16  SPO  advanced  this 
multinational  program  in  coxquncticm  with  the  Agile  Falcon.  Unlike  Agile  Falcon,  tiiis 
program  continues  to  survive. 

Tliis  program’s  eonneetiai  to  MSIP  is  quite  direct.  It  makes  a  series  of  changes 
already  engineered  fir  the  F-16G/D  under  MSIP,  many  of  whidi  are  at  a  nmilar  stage  of 
development  in  the  MSIP  effort  to  incorporate  tiiem  in  later  parts  ofF-16C/DBlodc  50.  The 
midrlifB  update  also  improves  the  rehabOity  and  operability  of  the  APO-66  fire-oontrol 
computer  in  tiw  same  way  that  MSIP  improved  the  APG-68  radar  fir  the  F-16(yD.  Andtiw 
update  integrates  these  changes  with  one  another  as  they  were  integrated  fir  the  F-16C/D 
under  MSIP. 

Ptadevek^nnent  continued  through  1969.  Full-scale  development,  whidi  is  sdwduled 
to  occur  during  1991-95,  will  cover  initial  fabrication  and  assemidy  test  retrofit  kits,  flight 
test  ofthoee  kits,  and  Lead-ttte-FleetoperatMWiel  testing  at  a  number  rf  installations. 
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DeUvay  of  production  kits  is  expedad  in  1996.  Until  this  program  is  executed,  we  cannot 
know  how  similar  it  is  to  MSIP;  eorrait  idana  suggest  strong  parallels. 

PfK)P08AL8  FOR  A  CLOSEWUn^UFPORT  ARCfUFT 

General  Dynamics  and  the  Air  Foroe  have  studied  a  doae-air-support  (CAS)  role  fin* 
theF-16.  They  have  developed  a  number  of  CAS  ooncqds  that  take  advantage  of  the  F-16*8 
ability  to  fly  doee  to  the  earth  atnudtt  and  designate  targets  with  a  laser,  capabilities  that 
could  be  improved  by  advances  in  digital  terrain  data,  terrain-fiollowing  and  -avoidance 
qrstems,  ni^t  vision,  and  target-definition  qrstems.  General  Dynamics  has  used  its  own 
fiinds  to  devel<9  and  validate  these  advances.  I&me  1987,  the  Air  Force  has  tested  and 
demonstrated  several  of  them  on  its  Advanced  Fif^ter  Technology  Integration  (AFTI)  F-16 
testbed. 

Proposals  differ.  In  1968,  the  Air  Force  developed  plans  to  buy  150  Blodc  50  F-16C/Ds 
a  year  modified  to  serve  the  CAS  In  1969,  the  NATO  commander  and  the  Air  Force  and 

Army  CSueCi  of  Staff  all  expressed  support  for  a  CAS-oriented  F-16  as  a  near-term  solution  to 
NATO's  perceived  need  fiir  CAS  aircraft  That  version  would  have  added  several  subssrstems 
to  new  Block  40  F-lOCVDs,  induding  an  improved  fire  control  radar,  GPS,  an  imjavved 
gunsi^t  end  the  potential  for  carrying  HABM  missiles.  A  variation,  considered  in  1989, 
would  have  upgraded  146  Block  30  F-lOs  to  the  CAS  role  by  adding  digital  terrain-following, 
ground-collision  avoidance,  an  automatic  target  hand-ofif  eystem  (ATHS),  a  30mm  gun  pod,  a 
PAVE  PEINNY  laser  tracking  pod,  and  armor.  In  1990,  the  Air  Force  proposed  the  A-16,  a 
variation  on  the  F-16CVD,  to  the  Defense  Acquisition  Board.  The  A-16  would  look  similar  to  a 
Block  50  F-16C/D  developed  under  MSIP,  and  it  could  potentially  be  managed  as  one  more 
tdodr  in  the  current  program,  efiectivdy  bringing  it  into  the  same  management  83rstem  as 
MSIP.  If  approved,  it  would  presumaMy  cimtinue  as  part  MSIP,  without  further  OSD 
oversi^t. 

All  these  proposals  share  a  common  feature:  use  of  an  MSIP  configuration  as  a 
baseline,  with  a  discrete  set  of  dianges  to  adneve  a  CAS  capability.  In  that  smise,  all  of 
them  like  a  new  Mock  in  MSIP.  Any  one  of  them  could  be  managed  in  a  niiwilar  way. 
lUs  pattern  of  proposal  is  testimony  to  the  power  of  MSIP  in  the  development  program  for 
the  P-16. 


SUMMARY 

The  F-16  development  program  remained  active  after  the  dose  of  its  initial  full-scale 
devdopmant  MSIP  was  the  dominant  devekqHnenteSbrt,  but  other  programs  played 
important  roles  and  took  advantage  oi  subeystems  tested  and  integrated  in  MSIP  and  of 
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iMMjM imifit  t*rhm<pi—  darwloparf  in  M«fP.  As  •  result,  all  sudi  ^Snts  share  ounnum 
dements. 

Placing  MSIP  in  the  amtext  of  these  other  dforts  helps  us  understand  that  the  MSIP 
approach  had  much  broader  impKcations  ftr  the  P-16  program.  All  these  programs 
i>— antislly  add  snbqrstems  developed  in  otiier  programs  to  a  hasdine  F-16  oonfiguration. 
The  programs  integrate  each  subqrstem  wHh  the  baseline  oonfiguration  and  with  all  other 
subsystems  being  added,  as  fidlows:  Thqr  simulate  the  pedbrmanoeofnewsubsiystems  in 
the  F-16  envinnment,  fdbricate  and  assemde  test  subsystems,  fli^t  test  than,  conduct 
operatumd  tests,  finally,  ineorporate  the  subsystems  into  the  induction  cS  a  new  F-16 

variation.  The  upgrade  afgiroadi  can  be  applied  to  new  aircraft  or,  through  retrofit,  to 
existing  aircraft. 

These  activities  «w«plui«i«s  the  usefulness  and  feasibility  of  modular  design  for 
subsystems  at  the  same  time,  the  effort  required  to  adapt  modular  systems  to  a  new 

configuration.  Although  *udi  efforts  are  deariy  devdoiunent,  test,  and  evduation  activities, 
♦hjiy  yyy  nffjtn  fiiTwbd  with  production  ftinda  as  nonrecurring  costs.  Therefore,  the  ongoing 
devd<qmient  effort  in  the  F-16  program  has  been  mudi  larger  than  a  simple  review  of 
researdi-and-devd<qHnent  funds  would  suggest 


5.  DCTiUtSONTHEF-ISIISIP 


The  preoediiig  seetums  explain  that,  althoti^  MSIP  was  a  distinct  program,  it  was 
(mly  part  of  the  F-16  program’s  continuing  itovelopmeDt  eflforts.  The  F-16  SPO  used  its 
normal  structure  to  manage  MSn*  as  part  ofthose  broader  efforts.  The  summary  of 
milestones  in  TaUe  5.1  places  MSIP  in  the  broader  context  of  the  F-16  program. 

With  this  perspective,  we  can  now  look  more  closely  at  the  management  goals  of  the 
F-16  MSIP  and  program  organization  by  the  F-16  SPO  and  General  Dynamics  to  achieve 
those  goals. 

In  particular,  we  want  to  answer  the  following  questiims:  What  specific  technological 
changes  did  MSIP  efEact  in  the  F-16?  What  risks  did  those  changes  present?  How  did  the 
F-16  SPO  and  General  Dynamics  manage  the  process  that  realized  those  changes?  What  was 
thcdr  basic  management  strategy  and  how  did  it  relate  to  the  risks  expected  in  MSIP?  What 
kinds  of  contracts  did  the  F-16  SPO  and  General  Dynamics  write  to  coordinate  their 

TaUe6.1 

Key  Events  Belevaiit  to  the  F-16  MHIP 


Year 

Key  Events  in  MSIP 

Key  MSIP-Related  Events 

1979 

MSIP  planning  begins 

F-16A/B  FSD  complete 

1980 

MSIP  Stage  I  begins 

F-16A/B  aduevee  IOC 

1981 

MSIP  Stage  n  begins 

First  MSIP  I  aircraft  ddivered 

600th  F-16  delivery 

1982 

F-16  Multiyear  I  awarded 

Falcon  Century  begins 

1988 

lOOOtfa  F-16  delivery 

1984 

First  F-16C  (MSIP  ff)  aiicraft 
delivered 

1966 

MSIP  Stage  m  begins 

1600th  F-16  deUvery 

1966 

F-16  Multiyear  n  signed 

F-16  selected  fbrADF 

1967 

First  F-16N  delivered.  Oparatioiial  C^>al»lity 
Upgrade  begins 

1988 

First  MSIP  m  (BloA  40) 
dfcraft  dsU'wrad 

First  F-16ADF  delivered 

1989 

F-16  Multiyear  in  signed 

1990 

Joint  FSX  design  team  begins 

1991 

Ffaet  MSIP  node  60  ahfcraft 

A_es -  m 

awfWM 

F-16A/B  Mid-Lilb  Update  FSD  b^ins 

1992 

RF-16  FSD  begins 

PfOMABTSOUBCB:  V  ft  Air  FimT,  Arrmirttryl  flyWine  Piriiiflii  F  Iff  nPTT  "Minstn 
aMBtlafcnsatiaaWotibook,FW6-2,*WT^ht-Pattw»oo  Air  Fow  Bast,  1»S6,  p.  0-3. 


activities?  How  did  those  omtracts  addnee  the  ridu  that  MSIP  presmted?  How  did  the 
contracts  change  as  MSIP  matured?*  This  section  addresses  each  of  these  questions  in  tom. 

THE  TECHNOLOGICAL  CONTENT  OF  CHANGES  EFFECTED  THROUGH  M8IP 

As  ATplttiiMwi  in  Section  1,  MSIP  has  proceeded  in  three  stages,  in  a  series  of  aircraft 
blocks  associated  with  these  stages.  The  program  is  set  up  to  allow  flexible  improvement  of 
the  F-16  as  suhsystems  beomte  available.  This  subsection  briefly  describes  the 

improvements  incorporated  in  Blocks  15  to  40  to  date  and  improvements  antidpated  in  the 
early  years  of  Block  50.  It  also  describes  briefly  the  level  of  risk  that  MSIP  managers  have 
associated  with  each  set  of  improvements. 

Stage! 

MSIP  began  finmally  in  February  1980,  when  detailed  work  on  Stage  I  was  authorized 
to  begin.  Because  that  work  required  little  new  design  or  development  effort,  it  was 
envisioned  that  the  changes  implemented  during  Stage  I  would  be  incorporated  in  F*16A/B 
aircraft  Nos.  330  throu|^  785  in  Block  15,  to  be  produced  starting  November  1981.  New  U.S. 
Air  Force  aircraft  were  affected  first;  changes  in  aircraft  destined  for  the  European 
Paitidpating  Governments  began  in  May  1982. 

Despite  such  short  lead  times  between  the  beginning  of  the  program  and  production 
delivery,  it  took  time  following  the  initiation  of  the  program  fin*  the  Air  Force  and  General 
Dynamics  to  agree  on  the  set  of  changes  to  be  induded.  A  highly  interactive  process  rapidly 
increased  the  number  of  changes  to  be  included  over  the  period.  In  the  end,  those  changes 
included  essential  structure,  wiring,  and  interfisce  provisions  to  support  future  aircraft 
avionics  changes  and  a  number  of  growth  systems.  Figure  5.1  summarizes  the  changes 
included. 

These  changes  added  no  immediate  combat  capability  to  the  F-16.  They  were 
essentiaUy  designed  to  reduce  the  cost  retrofitting  fiiture  systems  that  would  add  such 
capability.  Because  little  design  work  was  required,  developers  viewed  this  stage  as 
presenting  little  technical  risk;  rather,  the  main  risk  associated  with  this  stage  was  whether 
provisions  made  for  future  systems  were  the  right  ones.  If  future  needs  differed  fimn  those 
anticipated  during  Stage  I,  rework  would  be  required  to  retrofit  future  qrstems.  Any 

^Ihis  section  draws  heavily  <m  tbs  U.S.  Air  Force,  Aeranantical  flyeteme  Division,  F-16  Sjretem 
Regiam  Office,  1967  [hereinafter,  F-16  fVxyrowAfgaqgemciirPtothQwisralDynMaicB,  1990;  and 
Jane’s  Infiamatko  Onwv  limited,  varions  dates.  Riak  aseessmenta  in  particular  draw  heavily  on  the 
diaenaskm  of  indhridnal  MSIP  bio^  in  the  F-16  l^vgram  Mwiqgcment  Plan. 


•  Wing  structure  and  partial  wiring  provisions  for  beyoi^visual-range  air-to-air 
missiles 

•  Engine  inlet  structure  and  wiring  provisions  fiur  various  electro-optical  and  target- 
acquisition  pod  qrstems 

•  Cockpit  structure  and  wiring  provisions  for  a  wide-field-of-view  raster  HUD, 
multifunction  display  set,  data  transfer  unit,  and  Up  Front 
fVimmMnicetiaiis/Navigatian/Identification  (CNI)  system 

•  Wiring  provisions  for  an  expanded-capacHy  fire-control  computer,  advanced  weapons 
central  interface  unit,  radar  altimeter 

•  Early  structure  and  wiring  provisions  for  internal  electronic  countermeasures 
systems 

•  Increased-capacity  environmental  control  and  electric  power  s}rstems 

Flgiirr  B.1  — MfMHflratiffint  InfhiilTil  In  fftwgr  ^ 

surprises  associated  with  such  a  ri^  would  obviously  occur  in  future  stages  as  the  retrofits 
actually  implemented  as  part  of  MSIP  were  finalized. 

Stage  II 

Plans  for  the  second  stage  of  MSIP  were  presented  to  the  F-16  Multinational 
Cmifiguration  Steering  Group  in  October  1980.  Thqy  were  fimnally  authorized  in  May  1981, 
with  initial  production  deliveries  of  Block  25  aircraft  expected  in  December  1984.  The  first 
Blodk  25  F-16C  was  delivered  to  the  Air  Force  in  July  1984,  and  a  production  version  of  the 
F-18C/D  baseline  aircraft  was  realized  in  December  1984.  Production  of  increasing  capaUe 
Block  25  and  30/32  versions  of  Stage  II  aircraft  cemtinued  into  1989.  The  changes  induded 
during  this  stage  ooeuned  at  a  series  of  discrete  points  over  the  course  of  the  stage  as 
**miniMockB**  within  Blodcs  25  and  30  were  delivered  to  the  field.  These  pre-planned 
miniblockB  allowed  for  continuing  introduction  of  planned  changes  and  for  updates, 
paiticulaily  in  software,  that  were  found  to  be  desirable  as  MSIP  n  proceeded. 

This  stage  began  to  nmve  bqrond  the  existing  technological  base,  advancing  the  F-16 
program  from  the  F-16A/B  to  the  F-16C/D.  The  F-16  SPO  and  General  Dynamics  chose  to 
structure  MSIP  as  a  coordinating  environment  in  iddch  many  parallel  development, 
integration,  and  produetum  incorporation  activities  fiir  individual  subqystems  would  proceed. 
The  subaeetimu  below  <m  management  strategy  and  contraeta  say  more  about  this  approach. 


-36- 


The  second  stage  of  MSIP  provided  additional  aircraft  avionics  and  subsystem 
improvements  required  to  support  future  growth.  In  particular,  it  incorporated  subai^tems 
that  would  enaUe  a  aioj^e  pilot  to  perform  omnplex  tasks  associated  with  simultaneously 
flying  the  aircraft,  choosing  targets,  and  delivering  weiqwns  against  them  in  a  hi^-threat 
environment  Figure  6.2  shows  the  expanded  systems  that  were  included.  These  changes 
efiectively  began  to  fill  spaces  planned  for  in  Stage  I  and  to  continue  adding  capalnlity  to  add 
more.  They  substantially  increased  the  amount  of  infiMnnation  availaUe  to  the  pilot  and  the 
ease  with  which  that  information  was  used  in  combat  By  the  end  of  Block  30,  they  also 
added  new  weapon  capabilities. 


•  Wide-field-of-view  raster  head-up  display 

•  Multifunction  display  set  and  software-iaugrammalde  display  genesrator  to  replace  the 
then-current  stores  control  panel,  radar  display,  and  radar  symbol  generator 

•  Data  transfer  unit  that  allowed  the  use  of  a  cartridge  to  enter  mission  data  before  a  fli^t 

•  Up-firont  CNI  system 

•  Enhanced  fire-control  computer 

•  Advanced  central  interfooe  unit 

•  Radar  altimeter 

•  AN/APG-68  radsu:  incorporating  a  programmable  signal  processor  (PSP)  and  dual-mode 
transmitter  (DMT)  that  increased  the  range  and  resolution  of  the  radar,  and  the  number 
of  radar  modes  available;  improved  electronic  counter-countermeasure  capability;  and 
increased  flexibility  in  the  use  and  addition  of  modes  in  the  future 

•  Shrike  antiradiation  missiles 

•  Software  dianges  that  allow  full  level-IV  multitarget  compatibility  with  AMRAAM 

•  Ckmfigured  engine  bay  and  FlOO-FW-220  and  FllO-GE-100  Alternate  Filter  Engines, 
either  of  which  could  fit  in  the  bqy 

•  Modular  common  inlet  dnctdazge  forward  inlet  module  to  increase  airflow  to,  and 
fiierefore  ftall  available  thrust  fium,  the  FllO-GE-100  engine 

•  Structure  and  wiring  proviaimu  and  later  the  hardware  for  a  crash  survivable  fli^t  data 
recorder 

•  Improved  environmental-control-system  tuihine  assembly,  compatiUe  with  tbe  Stage  I 
environmeotal  system,  to  provide  aiUed  coding  air  capacity 


Figure  B  t  Modifications  Included  in  Stage  U 
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MSIP  managers  viewed  the  risk  asaodated  with  this  stage  as  low  to  mo<krate.  Most  of 
the  dianges  were  evotntkmary;  ftr  exampte,  the  greatest  management  attentum  among  these 
dianges  was  reeetved  bj  the  APO-68  radar,  whidi  "simply*  ad^  the  inogrammable  signal 
processor  and  dual-mode  tranamittw  to  the  APG-66  radar  already  in  service  on  F-16A/Bs. 
Similarly,  the  new  head-up  displays  evtdved  fhmt  a  head-up  display  already  in  service  on 
F-16  A/Bs.  The  Alternate  Fii^ter  EngiTies  were  derived  from  engines  already  in  service  on 
the  F-15,  F-16A/B,  and  B-IP  In  addition,  the  military  standards  used  to  plan  MSIP  were 
rasentially  those  used  to  pL  development  of  tlm  F-16A/B  or  standards  that  had  evolved 
fimn  tiietci. 

MSIPs  approach  to  managing  Stages  n  and  HI  introduced  an  additional  risk  during 
Stage  n.  For  reasons  discussed  in  the  subsection  "Management  Strategy*  below,  MSIP 
managers  placed  a  hi^ier  priority  on  meeting  the  schedules  of  Stage  III  than  those  of  Stage 
n.  As  a  result,  those  managers  expected  problems  in  Stage  n  resulting  firom  its  relatively 
low-priority  access  to  manpower,  simulation,  and  test  assets.  Althou{^  this  management 
approach  mi^t  have  increased  risk  in  Stage  II  of  MSIP,  it  was  not  expected  to  adversely 
affect  the  level  of  risk  in  the  program  as  a  whole. 

SlagaM 

Stage  m  continued  to  use  the  development-and-integration  approach  begun  in  Stage 
n,  including  the  continual  introduction  of  pre-planned  changes  and  updates  in  miniblocks 
within  Blocks  40/42  and  60/52,  which  currently  constitute  this  third  stage  of  MSIP.  Other 
blocks  have  been  considered;  for  example.  Block  50  is  a  scaled-back  version  of  an  earlier 
Block  70.  The  budget  cuts  that  led  the  F-16  SPO  to  reorganize  in  early  1987  also  forced  the 
F-16  program  to  restructure  Block  70  into  a  less  ambitious  Block  50  in  early  1987.  Blocks 
40/42  and  50/52  are  the  final  products  of  a  continuing  process  to  define  the  structure  of  MSIP 
m.  Authority  to  begin  Block  40/42  was  given  in  June  1985,  with  production  deliveries  of 
Block  40/42  aircraft  expected  to  begin  in  December  1988.  In  &ct,  MSIP  achieved  this 
milestone  on  schedule.  Blodr  40/42  production  ddiveries  are  scheduled  to  continue  into 
1992.  Preliminary  design  go-ahead  fin*  Block  50/52  came  in  Sqitember  1986,  with  initial 
aircraft  delivery  anticipated  in  June  1991.  Blo^  50  production  deliveries  are  now  expected 
in  October  1991. 

Stage  in  provides  for  installation  and  retrofit  of  specific  growth  systems  to  meet 
fitture  mission  needs,  systems  included  have  changed  repeatedly  as  information  has 
accumulated  on  the  technological  maturity  of  the  systems  considered.  Systems  included  in 
Block  40  were  structured  around  the  LANTIKN  system,  which  would  give  the  F-16C/D  new 
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terrain-foUowing  and  -targeting  capabilities  at  ni^^t  Figure  5.3  shows  the  systems 
introduced  to  date.  Other  systems  will  be  added  as  the  blodc  matures;  most  of  them  will  also 
appear  in  Block  50/52  aircraft  The  important  exceptions  are  the  LANTIRN  pods  and  HUD, 
which  effectively  define  Block  40/42  as  a  special  breed  of  F-16C/D. 

Figure  5.4  lists  modifications  expected  to  be  included  in  early  versions  of  Block  50/52 
aircraft.  These  additions  illustrate  how  the  development  approach  introduced  in  Stage  II  is 

•  LANTIRN  navigation  and  tainting  pods 

•  LANTIRN  dififiactive  optics  HUD 

•  APG-68V  fire-control  radar,  an  increased-reliabiliiy  modification  of  the  APG-68 

•  Aft-seat  HUD  monitor  in  the  F-16D 

•  Four-channel  digital  fli^t-control  system 

•  Enhanced-envdope  gun  si{^t 

•  Global  Positioning  S3r8tem  (GPS)  receiver  and  antennas 

•  Structural  strengthening 

•  Provisions  for  advanced  electronic  warfare  and  identification-fiiend-or-foe  (IFF) 
equipment 

Ffgure  5,3~-lMb>difieatioiu  Indnded  in  Stage  ID,  Btock  40/42 

•  Improved-performanoe  engines,  FllO-PW-229  or  FlOO-GE-129,  derived  firom  the  current 
Alternate  Fij^ter  Engines 

•  Advanced  programmable  signal  processor  using  VHSIC  technology  in  an  improved  APG- 
68V5  fire-control  radar 

•  HAVE  QUICK  HA  VHF  radio 

•  ALR-56M  advanced  radar  warning  receiver 

•  Provisions  for  the  automatic  target  hand-off  system  and  HAVE  SYNC  VHF  antijam  radio 
and,  later,  installation  of  these  systems 

•  Full  integration  of  HAR&f/Shrike  antiradiation  missiles 

•  Upgraded  programmable  display  generator  with  digital  terrain  system  provisions  and 
scope  for  digital  map  capability 

•  ALE-47  diaff  dispenser 

Figore  8A— Modiflcattons  Indnded  in  Stage  ID,  Block  80/52 
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contmtiing.  As  new  capabilities  become  availaUe,  tiiey  ate  programmed  for  integration  and 
production  incorporation.  Where  they  are  likely  to  reduce  tiie  cost  retrofit  witiiout  unduly 
increasing  risks  of  inemnpatilMlity  with  future  qrstems,  structure  and  wiring  provisions  enter 
the  aircraft  in  one  miniblock  in  anticipation  of  full  incorporation  of  the  related  hardware  in  a 
future  miniblock.  And  where  possible,  subq^stems  already  incorporated  in  the  F-16C/D  are 
allowed  to  improve  throu^  additional  subsystem-specific  development  ^ort 

MSIP  managers  associate  a  modmate  level  of  risk  with  Stage  m.  They  associate  low 
risk  with  the  integration  of  most  single  subssrstems  taken  alone,  but  expect  proldems  with 
the  coordination  of  concurrent  efforts  to  integrate  subs3mtems  with  the  baseline  airfirame  and 
to  integrate  many  developing  subsystems  with  one  another  as  they  mature.  The 
“Management  Strategy”  subsection  below  returns  to  this  problem  and  explains  why  these 
managers  believed  that  risk  associated  with  sdiedule  crald  be  bif^  for  many  of  these 
subsystems  and  for  MSIP  m  as  a  whole. 

Discussion 

MSIP  provides  for  progressive  enhancement  of  the  F-16.  Looked  at  solely  firom  the 
technology  “supply  side,”  this  approach  allows  the  F-16  to  benefit  firom  new  capabilities  as 
they  become  available.  Promotional  material  on  the  program  emphasizes  this  aspect  of 
MSIP  and  its  principal  product  to  date,  the  F-16C/D.  For  example,  in  its  F-16  program 
overview.  General  Dynamics  spends  its  opening  pages  on  the  F-16C/D  listing  all  the 
subssrstems  being  added  over  time  and,  in  its  first  direct  statement  about  the  aircraft,  says 

•  F-16C  Incorporates  Latest  Tedmology 

—Provides  Increased  Tactual  Ci^MbOity 
—Allows  Incorporatum  Emerging  Weapons  and  Sensor  Systems.^ 

But  MSIPs  progressive  approadi  to  improvement  also  offers  an  important  benefit 
from  the  designer's  “demand-side”  perspective.  Even  if  all  new  capabilities  were  available  at 
once,  a  progressive  series  of  introductions  enaUes  the  designer  to  sort  throu{^  selected  sets 
of  unknowns  at  a  time.  As  information  accumulates  on  subqrstems  introduced  early  and  the 
vray  they  wm^  together,  problems  wi^  these  subsystems  and  their  interaction  can  be  sorted 
out 

MSIP’s  miniUodE  system  enoouragad  sudi  an  m>proach  by  allowing  multiide  points  at 
whidi  to  introduce  improvemmts.  Asthedeeignforanint^trBtedsetofsubqrstems 
stabilised,  more  could  be  added,  beginning  the  process  of  infimnation  collection  and 


^General  Dynamics,  1960,  p.  29. 
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improvement  again.  We  see  this  ^>proach  most  directly  in  the  inonmental  development  of 
the  8(rftware  that  played  such  a  vital  nde  in  integrating  subeyrtems  with  the  airfirame  and 
one  another.  We  see  it  from  a  different  perspective  in  the  APG-68,  whidi  was  reintroduced 
in  a  new  and  improved  form  in  almost  every  new  blodc  of  MSIP. 

This  apineach  may  have  had  its  greatest  payoff  to  date  in  the  successful  development 
and  introduction  of  Block  40.  As  noted  above.  Block  40  was  specially  designed  around  the 
capabilities  provided  by  the  LANTIRN  system,  which  required  extensive  int^ration  of  the 
navigation  and  targeting  pods  with  conhrols  and  displays,  the  radar  altimeter,  terrain- 
avoidance  and  terrain-following  systems,  the  digital  fl^fot-control  ^stem,  and  air-to-ground 
weapons  on  the  aircraft.  (The  Appendix  details  this  integratioiL)  The  blocks  preceding  Block 
40  introduced  subeystems  required  to  support  these  new  capabilities,  although  these  earlier 
blocks  would  not  necessarily  be  retrofitted  with  LANTIRN  equipment  The  thoroii^  testing 
of  supporting  subsystems  in  earlier  blocks  limited  the  risk  associated  with  them,  setting  the 
stage  for  Block  40,  in  which  risk  reduction  efforts  could  focus  on  the  LANTIRN  and  the 
subsjrstems  most  closely  allied  with  it 

That  is  not  to  say  that  the  supporting  subs3rstems  introduced  before  Block  40  benefited 
only  Block  40.  In  fact,  they  also  set  the  stage  for  Block  50,  with  its  different  set  of 
subsystems  to  int^prate,  and  for  integrating  additional  capabilities  into  earlier  blocks  by 
means  of  retrofit. 

MSIP  has  the  appearance  of  introducing  myriad  improvements  as  individual 
int^prations,  and,  in  fact,  most  of  MSIP  has  been  organized  around  the  integration  of  specific, 
individual  subsystems.  At  the  same  time,  MSIP  itself  must  be  regarded  as  a  carefully 
planned  and  coordinated  environment  in  which  to  effect  such  integrations.  The  integrations 
required  to  realize  Block  40  were  carefully  structured  around  the  LANTIRN  system.  And 
each  set  of  integrations  has  built  on  the  sets  of  integrations  completed  earlim'.  Althou^ 
MSIP  is  designed  to  allow  continuing  improvement  in  the  design  of  the  F-16  by  integrating 
additi<mal  new  qrstems,  it  is  successful  in  doing  so  because  it  structures  the  way  the 
integrations  complement  one  another. 

MANAQEIIENT  STRATEGY 

The  idea  that  infrirms  MSIP  is  the  development  of  a  hi{^y  capable  aircraft  by  adding 
selected  incmnental  capabilities  to  an  existing  hi^-performance  design.  The  demand  for 
the  F-16  has  been  wdl  established  and  maintained  by  a  series  of  three  multiyear  contracts 
that  significantly  relieved  risks  associated  with  support  for  the  F-16  program  as  a  whole. 
Those  contracts  helped  free  the  F-16  SPO  and  General  Djmamics  to  focus,  in  MSIP,  on 
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managing  the  riaks  aaaodated  with  adding  eapabilitiae  to  the  F-16  in  the  of  the 

broader,  stable  demand  for  the  F-16  qrstem.  Because  the  filter  was  being  deveioped  and 
produced  for  many  users,  the  F-16  SPO  and  Gmieral  Dynamics  had  to  determine  a  process  of 
selecting  designs  for  new,  improved  F-16s.  And  because  capabilities  could  be  added  only  by 
adding  new  cystems  developed  elsewhere,  they  had  to  detennine  how  to  manage  the  risk 
associated  with  subsystem  designs  over  which  thqr  had  only  limited  oontzol.  Given  that,  aiqi’ 
new  design  would  incorporate  many  dmnges,  thqr  also  had  to  manage  the  risk  that  the 
systems  would  not  work  well  together.  And  because  MSIP  would  necessarily  be  handling 
many  tasks  at  once,  they  had  to  determine  how  to  manage  risk  associated  with  the 
concurrency  inherent  in  such  a  development  approadr  (mmn  if  no  ooncurrmicy  existed 
between  development  and  production).  'Diis  subsection  discusses  the  approach  that  the  F-16 
SPO  and  General  Dynamics  chose  for  handling  these  basic  management  issues. 

The  Stabmty  AasoeiaAad  with  MuMyear  Produetion  Contreeta 

When  the  F-16  SPO  and  General  D3mamics  conceived  MSIP  in  1979,  the  kind  of 
multiyear  inoduction  contract  that  has  become  familiar  was  not  feasible.  Sudi  contracts 
became  feasiUe  only  following  polity  changes  in  1981;^  therefore,  a  multiyear  contract  per  se 
clearly  played  no  role  in  the  early  planning  for  MSIP. 

But  the  F-16  became  one  of  the  first  systems  to  adopt  multiyear  contracting.  Its  first 
multiyear  contract  covered  FYs  82-85;  the  second,  FYs  86-89;  and  a  third,  FYs  90-93.  These 
contracts  point  to  a  stable  production  plan  in  a  healthy,  ongoing  weapon  program.  One  of  the 
koy  risks  associated  with  a  development  program — the  risk  that  the  program  will  not 
survive — is  likely  to  be  limited  in  such  an  environment. 

The  process  that  Congress  uses  to  approve  production  programs  for  multi3rear 
contracts  hi^ilights  a  variety  of  risks  and  approves  a  program  only  if  those  risks  are  limited. 
It  seeks  stalnlity  in  the  production  rate,  procurement  rate,  and  total  quantities  expected, 
continued  funding  over  the  course  cS  the  contract,  system  design  fior  the  portion  of  the  system 
covered  by  the  contract,  and  estimates  rf  eiq>ected  coat  The  fact  that  the  F-16  program  has 
exhilnted  the  stalality  required  to  maintain  multiyear  contracts  ever  since  they  became 
availaUe  suggests  that  similar  stability  was  present  even  befordiand,  during  earty 
discussion  of  MSIP,  and  that  dianand  fiir  F-16o  was  hi|dib^  likely  to  continue  over  the  course 
ofMSIP.^ 

>For  details,  see  Bodilly,  Caaun,  and  Pei,  1991. 

*OoB  might  argue  that  the  presence  of  the  multiyear  contracts  itself  directly  improved  the 
stability  ofthe  P-16  program.  WhUe  that  may  be,  ouch  a  owtraetltyiteelfcan  have  onty  limited  effects 
on  the  stability  of  a  manoiiectnring  program.  For  example,  even  with  a  multiyear  contract  in  place. 
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Given  such  stability,  the  challenge  for  MSIP  managers  is  to  design  a  way  to  build  a 
deTdopnimt  program  on  it  The  solution  that  MSIP  managors  reached  was  to  assume  that 
prodnctioai  of  F-16s  would  continue  regardless  of  progress  under  MSIP  and  to  absorb  much  of 
the  risk  associated  with  MfflP  by  changing  the  dates  at  which  new  capabilities  entered  the 
F-16  fleet  So,  fnr  ezamfde,  if  a  new  avionics  bos  was  ready  for  incorporation  a  year  later 
than  expected,  such  tardiness  need  not  delay  production  ofthe  system  as  a  whole.  Instead,  it 
could  be  introduced  a  year  later  than  expected  and  potentially  retrofitted  into  aircraft 
produced  in  the  meantime.  If  it  was  to  be  retrofitted,  structural  and  wiring  provisions,  or 
even  a  partially  complete  version  of  the  final  hardware  or  software,  could  be  implemented  on 
the  production  line,  limiting  the  cost  of  retrofit  when  it  did  occur.  Such  an  arrangement  was 
espedaUy  attractive  when  hardware  was  ready  on  time  but  software  was  late.  In  many 
cases,  software  could  be  updated  without  requiring  much  adjustment  to  the  hardware  in 
place. 

Such  a  strategy  obviously  has  limits.  MSIP  was  a  response  to  perceived  changes  in 
threat,  and  MSIP  managers  could  not  respond  effectively  to  that  threat  if  the  capabilities 
they  were  developing  were  delayed  too  long.  To  reflect  this  concern,  managers  picked  a  key 
milestone,  the  date  for  whidi  they  would  try  hard  to  maintain.  That  milestone  was  the 
introduction  of  Block  40,  the  first  installment  of  mqjor  new  capabilities  made  possible 
throu^  MSIP.  MSIP  managers  considered  that  maintaining  the  date  for  initial  production 
of  Block  40  aircraft  at  December  1988  was  their  most  critical  risk  in  the  MSIP  test  plan: 

The  goal  of  meeting  the  directed  F-16  operational  capability  associated  with  the  [Block 
40]  aircraft ...  is  critical.  This  milestone  cannot  slip  without  potentislly  significant 
impact  to  expected  operational  capability  and  retrofit  costs.  Schedules  and  technical  risk 
interact  heavily  in  meeting  this  milestone— both  within  the  ixulividual,  often  paraUel, 
development  efforts  and  the  final  integration  task  on  the  newly  configured  production 
aircraft.^ 


Introduction  of  selected  systems  could  dip,  but  the  basic  capabilities  required  fin:  Block  40 
would  be  hdd  to  that  date.  As  things  turned  out,  MSIP  could  not  quite  realize  this  goal  At 
the  December  1988  milestone,  hardware  dev^pmoit  and  testing  met  program 
requirements,  but  software  devdoianent  remained  incomplete,  and  ddiveiy  of  a  production- 
quality  software  tape  wouM  not  be  made  until  the  next  year.  Avionics  proUems  caused  Ity 

General  Dynamics  and  the  Air  Force  recently  agreed  to  a  nugor  reduction,  fiem  600  to  300  aircraft,  in 
the  inedndian  quantity  expected  lor  the  contract  nieunderiying  stability  ofthe  program  as  a  whide 
appears  to  be  more  important  than  the  presence  of  one  spedfic  contractual  dedoe. 

BF-16  Program  Managtment  Plan,  p.  5-8. 
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electromagnetic  interferenoe  from  sevval  avionies  boxes  persisted.^  And  althouid^  Blodc  40 
aircraft  were  capable  of  accepting  LANTIRN  pods  on  December  1988,  LANTIRN  pods  w«e 
not  ready  for  installation  by  that  time.  Tliey  were  installed  soon  aftmward,  however. 

Therefore,  althoue^  individual  improvements  could  slip  without  affecting  the  preset 
production  rate  for  ffie  underlying  F-16  airframes,  MSIP  would  be  set  up  to  work  around  that 
rate.  Stability  in  the  production  program,  of  course,  could  not  guarantee  stalnlity  in  the 
development  program.  And,  in  foct,  MSIP  went  through  many  changes  during  its  first 
decade.  Flexibility  is  a  central  characteristic  of  the  program.  But  a  stable  production 
program  could  limit  risks  that  mij^t  endanger  the  development  program  in  the  absence  of 
stable  production. 

One  Development  Program  for  Many  Users 

General  Dynamics  has  sold  the  F-16  to  19  different  governments.  Each  typically 
wants  a  somewhat  different  design,  and  some  want  more  than  one  design.  Even  if  the 
general  production  rate  is  eiq)ected  to  be  stable,  MSIP  must  adapt  its  planning  process  to 
meet  the  needs  of  so  many  users,  especially  when  those  needs  are  likely  to  change  over  time. 

The  key  to  this  problem  is  the  predominance  of  the  U.S.  Air  Force  among  General 
Dynamics’  customers.  MSIP  has  effectively  served  the  U.S.  Air  Force  first,  testing  and 
delivering  U.S.  configurations  first  When  budget  cuts  or  shortages  of  test  assets  have 
threatened  the  program,  MSIP  has  tended  to  place  priority  on  U.S.  interests.  And  in  the  end, 
MSIP  developed  a  large  set  of  capainlities  that  the  U.S.  Air  Force  wanted,  essentially 
creating  a  menu  that  other  countries  could  choose  firom  when  customizing  their  own  designs. 
The  basic  MSIP  plan  covered  all  the  wmrk  required  to  int^pnte  a  new  eystem  with  the 
baseline  F-16  and  most  of  the  woHt  required  to  integrate  any  set  of  subeystems  with  one 
another.  Additional  work  required  to  cmnidete  integration  on  a  customized  version  was 
small  relative  to  the  program  as  a  wlude.  Hence,  the  U.S.  Air  Force  could  take  primary 
responaifaility  for  MSIP  without  seriously  compromising  the  interests  of  potential  foreign 
buyers. 

This  is  not  to  minimize  the  role  foreign  bqjrers  played  in  MSIP.  On  the  contrary, 
designs  routing  went  through  the  Multinational  Configuration  Steering  Group  for 
approval,  effeetivdy  invdving  the  European  Particqwting  OovwrwnMint*— tuiginm, 

Denmark,  Tlte  Netherlands,  and  Norway.  Security  restrictions  limited  transfer  of  some 
infiirmatimi  and  ddayedreleaM  of  other  information  in  the  process.  But  the  multinational 
partners  in  the  original  F-16  program  remained  active  throu^umt  the  process.  And 


•Wolf;  1968,  pp.  184-186. 
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accommodato  needs  imposed  by  short  Norwepao  nmways,  Nwwegisn  F>16s  were  nwMitfUiH 
to  add  a  ftiring  fbr  a  braking  peradmte  to  the  base  of  the  vertiesl  fin. 

In  addition.  Istad  maintained  a  development  progTMBofits  own  that  coordinated 
testing  with  MSIP.  The  program  enabled  Israd  to  add  omiabiltties  to  the  MSIP*generated 
F-16b  it  received  that  were  not  availidde  elsewhere  in  the  fleet  Bntlsrad’scmimitmenttoa 
serioos  complementary  develo|nttent  program  is  unique  among  F-16  users. 

The  dominant  pattern,  then,  was  for  the  U.S.  Air  Force  to  create  a  menu  of  new 
capabilities  that  other  nations  could  dumae  firom.  The  success  of  this  apinoach  may  hdp 
a»pliiin  foreign  buyers*  concern  that  the  U.S.  Air  Force  would  lose  interest  in  the  F-16A/B 
and  thereby  reduce  the  benefits  the  Air  Force  created  by  playing  a  central  role  there.  It 
pliQied  such  a  role  throu^umt  the  MSIP. 

Canltaliiina  on  CafMblMiaa  OavoiODad  Elaawhara 

The  heart  of  the  MSIP  approach  is  the  incorpmation  of  new  ciqmbilities  into  the  F>16 
design  as  thqr  become  availaUe.  With  a  few  exceptions,  contractors  other  than  General 
Dynamics  or  its  subcontractors  developed  the  mqjor  subsystons  used  in  MSIP,  and  SPOs 
other  than  the  F>16  SPO  oversaw  the  development  of  those  subsystems.  Rdying  on 
teduKdogy  sources  beyond  the  immediate  reach  of  the  F>16  SPO-General  Dynamics  nexus 
raises  a  number  of  risks. 

The  first  challenge  is  knowing  what  capabilities  exist  and  what  risks  mi^t  be 
associated  with  them.  The  F-16  SPO  set  up  the  Falcon  Century  Program  in  1682  to  assess 
technologies  that  mi^t  be  incorporated  in  BISIP  in  the  future.  The  Falcon  Century  Program 
evaluates  future  development  and  production  alternatives  and  links  the  availability  of 
evolving  tedinologiee  to  required  mission  capabilities.  It  evaluates  alternative  design 
configurations  against  sudi  identified  requirements  as  weapon  system  performance, 
production  and  retrofit  feasibility,  cost,  and  mission  efiectiveness.  Falcon  Cmtory  serves  all 
programs  in  the  SPO,  not  just  MSIP,  but  it  has  played  an  important  part  in  long-term 
planning  on  later  blocks  of  MSIP. 

Once  BubeysteuM  are  identified  fisr  a  potential  rde  in  MSIP,  relatuniships  must  be 
established  To  iqien  communication  with  the  relevant  groups,  the  P-16  SPO  generated 
martymemoraiida  of  agreement.  Figure  6.5  lists  those  in  place  in  1987.  The  organixations 
invulvad  indude  a  large  number  (rf' other  SPOs,  govemmoit  laboratories  and  othff 
government  agencfes.  Table  5J2  summarises  the  F-16  SPO’s  sharing  of  responsibility  with 
other  SPOs  fbr  the  mnjor  subsystems  induded  in  MSn*.  Other  SPOs  generally  retain 
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responribility  for  the  development  and  teat  of  each  subayatem.  Many  alao  retain 
reaponsifaility  for  imxluction  and  aupport  of  Group  B  itema  (the  aubayatema  without  the  parts 
required  to  integrate  them  with  the  F-16,  which  tq>pear  in  Grmip  A).  The  F-16  SPO  takes 
responsibility  for  moat  other  activities.  (Smieral  Dynamics  has  similarly  signed  associate 
contractor  agreements  with  firms  planning  to  provide  government-furnished  equipment  that 
MSn*  would  int^rate  into  the  F-16CVD.  To  coordinate  the  activities  of  the  mqjor  contractors 
involved,  interfiace-control  working  groups  also  exist  for  each  nu^  subsystem. 

The  next  chAllange  is  technical  compatibility.  Modular  design  maintains  flexibility,  for 
the  <t««rigner  and  the  potential  user,  but  a  subsystem  must  ultimately  be  integrated  with  the 
systems  of  which  it  will  become  a  part.  The  F-16  SPO  and  General  Dynamics  faced  this 
problem  by  trying  to  get  involved  eariy  in  development  programs  for  the  subs3rstems  of 
interest  At  the  very  least  throucd*  the  agreements  above,  they  exchanged  information  on  an 
ongoing  basis.  They  often  sou^t  a  more  active  role.  The  F-16  SPO  played  an  active  role  in 
defining  the  performance  requiremmts  for  a  number  of  new  systems  as  early  as  their 
concept-validation  stages.  The  F-16  SPO  provided  aircraft  as  testbeds  for  selected  qrstems, 
fortuitously  allowing  them  to  begin  preliminary  integration  with  an  F-16  system  early  in 
their  full-scale  development  and  sometimes  earlier.  Similarly,  General  Dynamics  permitted 
access  to  its  principal  simulators,  the  Systems  Integration  Laboratory  (SIL)  and  MSIP 
simulator.  Developers  could  test  software  in  the  complex  information  environment  m  which 
it  would  have  to  operate  well  brfore  hardware  was  ready  for  testing  in  other  than  a 
simulated  environmoit.  The  Appendix  illustrates  many  such  activities. 

Of  course,  int^ration  and  testing  would  continue  after  the  subsystem  itself  was  fully 
developed.  The  sulnystem  would  be  checked  in  the  Systems  Integration  Laboratory,  undergo 
functional  hardware  and  software  tests,  and  then  undergo  flight  tests.  Such  tests  rarely 
proceeded  successfully  the  first  time.  MSIP  managers  routinely  used  such  testing  to  isolate 
proUems  that  required  further  ^elopment.  Repeated  testing  and  development  pursued  a 
TAF  qpde  that  played  an  important  part  in  bringing  all  subq^stems  into  the  broader  F-16 
environment. 

Given  the  iqipUcabiUty  of  a  teat-analyze-fix  approach,  it  is  useful  to  find  ways  to 
integrate  the  sobqystem  and  an  F-16  surrogate  td  some  kind  as  early  as  possible.  One  way  to 
do  so  is  to  aedc  condination  eaity  in  the  development  of  the  subqratem  and  use  the  F-16 
actively  in  the  testing  of  the  aystem.  This  obviously  creates  a  high  degree  (d'amcurrency 
between  the  devdopment  of  the  sub^rstem  and  integration  efforts  in  MSIP.  Such 
concarrenqr  has  been  common  in  MSn*.  Figure  5.6  illustrates  this  approach  by  showing 
MSIFs  view  of  the  AMRAAM  program  in  1686,  which  indicates  MSIP  flight  tests,  missile 
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Figure  S,S>-MSIP  Maater  Sehednle  for  AMRAAM  Integration  with  the  F*16C/D 


shots,  and  production  incorporation  of  the  system  are  occurring  well  before  full-scale 
development  (FSD)  for  AMRAAM  itsdf  was  scheduled  to  end  in  1988.  That  MSIP  did  not 
realize  thin  sdiedule  with  AMRAAM  doM  not  negate  the  fitct  that  it  was  attempted.  MSIP 
executed  similarly  concurrent  schedules  with  many  other  developing  subsystems. 

Relying  on  sobs3^stems  still  in  devdopment,  of  course,  presents  risks  of  its  own.  There 
is  no  guarantee  that  developing  subsystems  will  achieve  their  stated  goals  or  achieve  those 
goals  on  a  sdiedule  compatible  with  that  fm  IdSIP.  MSIP  often  balanced  risks  by  seeking 
subsystems  being  derived  from  other  subsystems  with  known  performance  characteristics,  so 
that  MSP  could  influence  those  tytteau  during  their  development  to  improve  compatibility 
with  the  F-16  and  to  take  advantage  of  recent  improvements  in  technology  without  accepting 
the  risks  associated  with  msjor  new  developments.  That  is,  just  as  MSIP  was  itself  a 
derivative  development,  it  often  soui^t  subsystems  that  were  derivative  develi^ments 
themselves.  Combined  with  the  test-analyze-fix  approach,  this  strategy  allowed  MSIP  to 
exploit  a^iarently  modular  developments  while  amdiorating  the  risk  that  th^y  would  yield 
flnal  gystons  incompatible  with  the  F-16.  And  when  a  sulteystem  program  was  uneaqiectedly 
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delaysd,  particolarly  a  software  pit^pam,  MSIP  could  often  absorb  such  a  delay  without 
endangering  the  key  December  1988  milestone  for  its  total  Block  40  ^stem. 

bitagraling  Savaral  Naiw  CapabUittas  Sbnuttanaously 

For  the  most  part,  MSIP  managers  eapected  the  int^ration  of  individual  subsystems 
to  present  little  difficulty.  They  associated  a  low  risk  with  most  subsystems  and  rarely  more 
than  a  moderate  risk  with  selected  subsystems.  Integrating  all  of  them  at  the  same  time 
introduces  additional  risks  that  led  some  managers  to  judge  the  schedule  risk  as  hi^  for 
MSIP  as  a  whole.’'  The  simultaneous-integration  problem  has  two  components;  having  a 
flight-tested,  final  configuratimi  available  at  all  times  to  meet  the  production  schedule;  and 
testing  aarfi  subsystem  in  a  final  configuration  still  under  development,  a  configuration  that 
win  still  be  incomplete  until  each  subsystem  is  finalized. 

Unless  a  final  configuration  has  a  constellation  of  subqrstems  that  allows  safe  lli^t,  it 
cannot  be  delivered  and  fielded.  If  all  subssrstems  complete  their  development  and 
integration  programs  on  schedule,  the  final  configuration  can  be  delivered  as  planned.  If 
even  one  subsjrstem  is  unavailable,  the  final  configuration  must  be  delayed  or  a  substitute 
must  be  found  for  the  missing  subsystem  so  that  a  substitute  final  configuration,  presumably 
missing  some  desired  capability,  can  be  delivered.  To  maintain  the  production  rate  dictated 
by  the  underlying  production  contracts,  MSIP  has  not  wanted  to  delay  delivery  under  any 
drcumstances.  Hence,  it  has  sou^t  ways  to  provide  substitutes  for  subsystems  that  have 
not  completed  development.  As  a  final  configuration  attempts  to  integrate  more  new 
subsystems  at  a  time,  the  probability  of  this  problem  occurring  rises.  And  the  prolmbility 
that  more  than  one  subsystem  will  be  missing,  yielding  a  delivered  F-16  with  still  less 
capability  than  expected,  also  rises. 

MSIP  plans  for  this  problem  in  three  ways.  First,  it  tries  to  get  involved  early  in 
individual  subsystmn  programs  to  estaUish  the  status  of  those  programs  and  the 
perfinrmance  requirements  for  each  subsystem.  The  earlier  the  perfiinnance  requirements 
can  be  established  for  each  subsystem,  the  easier  it  is  to  test  other  systems  and  the  F-16  as  a 
whole  to  ensure  that  all  other  parts  of  the  sj^stem  are  compatible  with  those  requirements. 
When  several  subsystems  develop  simultaneously,  early  requirements  definition  for  each  of 
them  specifies  a  nominal  total  sjrstem  that  all  can  move  toward  as  they  mature.  It  reduces 
uncertainty  about  the  environment  that  each  will  encounter  at  maturity  and,  even  when  not 

managers,  fiir  ezamide,  judged  Blocks  40  and  60  to  inesent  moderate  to  high  risks, 
primarily  because  of  sadiooncumncy.  F-16  Pnogiwn  ifanugemeiU  P(oa,  pp.  15-2, 15-3. 
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all  requiranenta  are  mat,  limits  the  final  adjustments  required  to  bring  all  individual 
subsystems  together. 

Second,  if  a  test-analyse-fiz  cycle  makes  sense  for  individual  subqrstems,  it  makes 
even  more  sense  for  groups  of  them.  Simulation  and  modeling  have  improved  a  developer’s 
ability  to  predict  and  resolve  problems  associated  with  well-understood  processes.  As  more 
subsystems  must  be  coordinated,  however,  their  joint  performance  becomes  inereasinfi^y 
difficult  to  model  analytically.  Hence,  the  empirical  tests  in  a  test-anal]rze-fix  cyde  are 
indispensable  to  integration  ^orts.  MSIP  reflects  this  insist  by  anticipating  a  need  to 
iterate  attempts  at  integration.  At  preset  times,  MSIP  tests  those  aspects  of  individual 
subsystems  that  <*-en  be  tested  then  for  their  fit  with  other  subsystems.  The  Systems 
Integration  Laboratoiy  plays  an  int^ral  part  in  this  process  because  it  can  test  important 
aspects  of  new  systems  fairly  early  in  the  integration  process.  Early  iterations  invariably 
identify  integration  problems  that  may  require  adjustment  in  the  subsystem  being  added  or 
in  some  other  part  of  the  system  that  mi^t  initially  have  appeared  to  have  no  connection  to 
the  subsystem  in  question.  Early  iteration  also  helps  identify  serious  integration  problems 
early,  giving  developers  more  flexibilify  in  finding  a  solution.* 

One  solution  may  be  to  replace  a  subsystem.  This  is  the  third  way  developers  plan  for 
the  problem  of  a  poorly  performing  subsystem.  As  problems  emerge  in  the  integration 
process,  developers  begin  to  seek  alternatives.  The  alternative  may  be  the  subsystem 
currently  in  the  baseline  F-16;  an  improved  version  of  the  fidling  subsystem,  i.e.,  the 
subsystem  has  experienced  further  development;  or  another  subsystem  entirely.  In  some 
cases,  developers  have  used  one  subsystem  as  a  short-term  fix  while  continuing  development 
to  get  the  performance  that  they  really  want  firom  a  subs3rstem  at  a  later  introduction  date. 
Until  such  a  subs3rstem  is  introduced,  however,  a  substitute  must  be  found  to  allow  any  final 
configuration  to  fly.  This  aspect  of  the  integration  process  emphasizes  the  importance  of  ties 
between  MSIP  and  the  development  programs  for  the  subsystems  that  it  uses.  It  also 
emphasizes  the  importance  to  maintaining  contacts  with  other  development  programs  well 
into  the  integration  process.  On  more  than  one  occasion,  MSIP  reopened  the  development 
process  fiir  a  subssrstem  by  the  fbiling  subsystem  back  into  a  full-scale-development 

competition  with  an  alternative  subsystem.  In  at  least  one  case,  that  fbr  the  radar  warning 
receiver,  the  alternative  ALR-56M  replaced  the  ALR-74  originally  induded  in  MSIP  tiirou^ 
a  renewed  competition  and  fly-off. 

*  MSIP  focuses  on  integratkmaetivitMB,  not  the  testing  of  sobqnUans  perse.  Butteststo 
integrate,  say,  a  jammer  into  the  F-16  could  rim|Jify  future  effivts  to  integrate  that  same  jammer  into 
other  aircraft.  In  this  way,  IdSIP  could  be  seen  as  sunxirting  underlying  subsystem  development 
programs. 
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AD  these  problems  address  the  need  to  integrate  subsystems  to  allow  delivery  o£  a 
final  configuration.  This  is  the  most  visible  multisystem  int^ration  problem  that  MSIP  has 
faced.  But  another,  closely  related  problem  has  also  presented  serious  difficulties.  If  a 
subsjrstem  is  not  available  to  comply  a  final  configuration,  it  is  also  not  available  to  test 
other  sub^stems  in  the  configuration.  The  unavailability  of  subs^tems  fc/  use  as  test 
assets  has  repeatedly  delayed  test  and  integration  of  other  subsystems.  MSIP  devdlopers 
have  reacted  by  Hiring  every  means  available  to  maintain  schedules.  For  example,  they  have 
pressured  the  developers  of  the  mining  assets  to  deliver  them,  tested  features  that  could  be 
tested  in  the  absence  of  the  misiring  assets,  used  simulation  or  analysis  to  infer  what  tests 
might,  have  revealed  if  they  could  have  been  conducted,  and  substituted  alternative  assets 
with  irimilar  features  to  make  other  inferences.  This  type  of  problem  repeatedly  has  tested 
the  imagination  of  a  creative  manager.  The  Appendix  provides  examples  of  this  problem. 

The  F-16  SPO  and  General  Dynamics  obviously  cannot  do  all  these  testing  and 
integration  taaka  by  themselves.  Active  involvement  of  other  SPOs  and  contractors  is  vital. 
Memoranda  of  agreement  and  associate  contractor  agreements  facilitate  coordination.  By 
early  1961,  an  F-lb  Int^ration  Executive  Committee  was  established,  with  membership 
fiom  the  AMRAAM,  ASPJ,  LANTIRN,  GPS,  PLSS,  JTIDS,  and  SEEK  TALK  SPOs,  as  weU 
as  Headquarters,  Air  Force,  the  Air  Force  Systems  and  Logistics  Commands,  and  the 
Tactical  Air  Command.  It  would  later  become  the  MSIP  Executive  Committee.  It  set  up  five 
working  groups  to  cover  technical  integration,  test  planning,  logistics,  production  planning, 
and  finance.  By  the  same  time,  meetings  ware  being  held  with  representatives  of  the 
LANTIRN,  PLSS,  GPS,  AMRAAM,  ASPJ,  SEEK  TALK,  and  JTIDS  SPOs  to  establish  master 
development  and  integration  test  schedules,  optimize  use  of  F-16  test  aircraft,  and  identify 
test  hardware  shortfidls. 

No  matter  how  much  interaction  occurred,  however,  Gmieral  Dynamics  retained  the 
standard  total-eystem-performance  responsibility  that  one  would  expect  of  a  prime 
contractor.  And  given  the  nature  of  MSIP,  the  Air  Force  required  that  definitized  production 
contracts  contain  an  Integrated  System  Performance  Responribility  (ISPR)  provision  as  well. 
This  provision  states  that  General  Dynamics  is  responsible  fin:  ensuring  that  selected  nu^ 
integrated  qrstems,  including  govemment-fumished  equipment,  meet  the  perfimnance 
requirements  defined  by  the  Air  Vehide  Specification,  provided  that  all  govemment- 
fhmidied  MSIP  subsystems  meet  their  individual  performance  requirements. 

To  perform  its  role  in  overseeing  int^pation  effints,  the  F-16  SPO  turns  to  its 
directoFateofengineering  for  technical  expertise.  This  directorate  has  established  special 


procediires  for  addressing  integration  problems  associated  with  the  following  subsystem  and 
engineering  programs:^ 

•  environmental  control  systems 

•  electrical  power  system 

•  aircraft  structural  integrity  program 

•  corrosion  control  program 

•  aircraft  weight  summary  and  maximum  gross  weight 

•  volume/equipment  locations 

•  landing  gear 

•  aircraft  performance 

•  radio  firequeni^/electromagnetic  compatibility. 

Many  sub^stems  are  clearly  in  direct  competition  for  the  services  of  the  aircraft’s  central 
environmental  control  and  power  systems  and  for  the  limited  space  and  weight  available  on 
the  aircraft.  Many  also  compete  for  use  of  a  limited  set  of  radio  frequencies.  Subsystems  can 
have  unexpected  effects  on  the  structural  integrity  or  effective  lifetime  of  other  sub^stems; 
corrosion  control  presents  a  similar  problem. 

Managing  Muitipla  Oemanda  on  Test  Resources 

MSIP  would  ultimately  have  to  address  the  concerns  above  in  a  concrete  test  program. 
MSIP  managed  its  test  program  through  the  standard  structure  of  the  F-16  SPO  and  related 
organizations.  The  SPO  directorate  responsible  for  testing,  which  changed  over  time,  had 
primary  responsibility.  Its  most  difficult  time  during  MSIP  began  in  1987,  when  budget  cuts 
severely  reduced  its  test  management  expertise.  This  directorate  worked  closely  with  the 
3246th  Test  Wing  at  Eglin  Air  Force  Base,  the  F-16  Combined  Testing  Force  maintained  by 
the  Air  Force  Flight  Test  Command  at  Edwards  Air  Force  Base,  and  General  Dynamics. 
Together  with  the  Air  Force  Operational  Test  and  Evaluation  Command  (AFOTEC)  and  the 
Tactical  Air  Command  (TAC),  these  organizations  composed  the  Test  Management  Council 
that  met  quarterly  to  coordinate  test-related  interaction  among  these  organizations.  TAC 
played  a  special  role  in  this  organization,  both  as  a  source  of  aircraft  to  be  used  in  flight  tests 
and  as  the  source  of  priorities  on  what  aspects  of  system  performance  deserved  emphasis 
when  not  all  testing  goals  could  bo  realized. 


®F-16  Program  Management  Plan,  pp.  4-8-4-10. 
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Fot  MSIP,  these  organizations  coordinated  a  test  program  and  an  associated  test  and 
evaluation  master  plan  (TEMP)  to  identify  risks  and  structure  a  test  plan  to  address  those 
risks.  To  obtain  OSD  approval  for  the  Block  40  test  plan,  they  also  assembled  a  baseline 
correlation  matrix  that  documented  relationships  between  mission  requirements  and 
evaluation  criteria  to  ensure  coordination  between  development  and  testing.  Within  MSIP, 
testing  affected  the  integration  of  individual  subesrstems  and  assessment  of  full  F-16C/D 
configurations.  Each  system-integration  program  maintained  its  own  TEMP.  MSIP  then 
maintained  a  test  program  separate  from  these  individual  efforts  that  focused  on  the 
F-lgCVD  aircraft  itself  and  tested  it  as  a  system  to  qualify  new  configurations  for  production. 
This  test  program  focused  on  the  carriage  and  separation  envelope  for  aU  MSIP-related 
stores  and  munitions  loadings  and  on  the  effect  of  MSIP  equipment  on  F-16  performance, 
flying  qualities,  and  structural  considerations.^*^  Such  a  test  program  obviously  had  to  rely 
heavily  on  the  tests  run  on  individual  subsystems  and  their  individual  integrations  into  the 
F-16C/D.  To  facilitate  coordination  among  all  the  test  programs,  the  MSIP  Executive 
Committee  set  up  an  MSIP  Test  Planning  Subgroup  composed  of  ‘ley  test  and  support 
equipment  managers  fix>m  the  various  program  offices,  test  centers,  the  users,  AFOTEC,  and 
contractors.”^^  Table  5.3  lists  the  k^  participating  organizations  in  this  group  in  the  mid- 
1980s.  Membership  changed  over  time  as  the  key  systems  included  in  MSIP  changed. 

This  subgroup  identified  a  set  of  testing  issues  and  risks  specificaUy  relevant  to  MSIP. 
Concern  about  maintaining  the  December  1988  milestone  for  initial  Block  40  delivery, 
mentioned  above,  was  their  primary  concern.  Other  concrams  dealt  more  with  the 
coordination  of  resources  used  in  testing  and  of  data  generated  by  testing;  they  included: 


TaMe6,8 

Key  Meadwr  Oiganizations  in  the  MBIP  Test  Planning  Subgroup 


Program 

SPO 

RTO/PTO 

Contractor 

F-16 

ASIVyPD 

APFTC/AD 

General  Dynamics 

LANTIRN 

ASD/RWN 

AFFTC/TBD 

Martin  Marietta, 

Marconi 

AMRAAM 

AIVYM 

AD/APFTC 

Busies 

ASPJ 

NAVAIR/PMA-272 

AD/AFFTC 

Westin^iouse/ITT 

OPS 

8D/YE 

AFFTC/ATBD 

Magnavox,  RockweU 
Collins 

ALR-74/ALR-56M 

ASIVYPD 

AD/TBD 

^plied 

Technology/Loral 

SOURCE:  P-18  Program  Manogement  Plan,  p.  6-7. 


lOp-lS  Program  Management  Plan,  p.  5-11. 
^V-16  Program  Management  Plan,  p.  5-7. 
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•  co-use  scarce,  hi^  cost  test  assets  for  both  development  and  integration  tasks; 

•  adequate  funding  for  test  and  evaluation  assets,  including  contractor  support 
where  required; 

•  adequate  and  consistent  priorities  at  the  various  test  sites  and  within  the  various 
test  programs;  and 

•  cooperative  development  of  integrated  test  objectives  and  the  co-use  of  related 
testdata.^ 

Among  these  concerns,  access  to  F-16  test  aircraft,  coordination  of  the  available  F-16  test 
aircraft,  access  to  the  Genmul  Dynamics  Systems  Integration  Laboratory,  and  access  to 
prototype  subsystems  that  could  be  used  in  test  configurations  presented  the  greatest 
challenges  in  the  development.  Budget  shortfalls  in  the  late  1980s  also  created  difficulties. 
The  Appendix  presents  examples  specific  to  these  concerns. 

Discussion 

As  a  successful  program  perceived  to  present  limited  risks  in  its  production  program, 
the  F-16  program  was  attractive  for  conducting  a  development  program  like  MSIP.  The 
MSIP  management  strategy  limited  risks  associated  with  the  program  as  a  whole  and  made 
it  easier  for  developers  to  plan  for  and  react  to  risks  associated  with  development  itself. 
Within  that  strategy,  developers  dealt  with  the  problem  of  having  potentially  many  users  by 
giving  a  single  user,  the  U.S.  Air  Force,  priority,  and  essentially  offering  other  users 
variations  that  could  be  constructed  firom  options  included  in  the  U.S.  Air  Force  program. 
They  dealt  with  risks  associated  with  integrating  individual  subsystems  developed  elsewhere 
by  getting  involved  in  the  development  of  derivative  subsystems  early,  learning  about  their 
capabilities,  and  shaping  their  development  to  promote  integration  with  the  F-16.  Early 
involvement  enabled  them  to  use  a  test-analyze-fix  cycle  to  promote  integration.  They  used 
similar  techniques  to  deal  with  risks  associated  with  integrating  several  subsystems  at  once. 
Such  risks  were  more  difficult  to  plan  for  and  required  especially  creative  reactions  by 
managers  to  keep  the  pn^ram  on  track. 

Hm  concurrency  MSIP  managers  experienced  as  thqy  coordinated  early  with  the 
development  pi  grams  for  subssrstems  and  integrated  several  sub^stems  simultaneously 
imposed  the  most  severe  residual  risk  in  the  program,  a  risk  important  enou^  for  some 
managers  to  believe  that  the  MSIP  fiued  a  hi^-risk  sdiedule.  Such  risks  existed  in  part 
because  the  developers  chose  to  meet  shortfalls  by  letting  the  schedule  for  final  production 


^F-16  Program  Management  Plan,  p.  6-8. 
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incofporations  slip.  Such  an  approach  was  compatible  with  the  underlying  production 
program.  But  to  ensure  timely  delivery  of  the  1^  system  in  the  MSIP,  the  fully  capable 
Rfga  m  F-16C/D,  BiSIP  tried  to  hdd  &8t  to  the  first  delivery  of  Stage  III  F-16C/Ds.  These 
F-16s  were  delivered  without  all  the  anticipated  capalnlities,  but  an  orderly  program  of 
retrofits  has  restored  those  capabilities  as  th^  have  become  available  to  MSIP. 

CONTRACTS 

Although  many  organizations  have  contributed  significantly  to  MSIP,  the  key 
relationship  in  the  F-16  MSIP,  as  noted  above,  has  been  that  between  the  F-16  SPO  and  the 
F-16  contractor.  General  Dynamics.  The  relationship  between  the  two  organizations  has 
governed  the  planning  and  execution  of  the  MSIP  development.  To  define  that  relationship 
Hnr-ing  MSIP,  the  F-16  SPO  and  General  Dynamics  used  a  series  of  contracts.  Those 
fAntra*^  listed  in  Table  5.4,  give  the  very  strong  sense  of  having  evolved  finm  one  into  the 
nfhar  as  the  program  itself  matured.  That  is,  as  the  program  became  increasin^y  well 
HftfinaH,  contracts  better  tailored  to  the  needs  of  the  program  could  be  devised. 

MSIP  haga"  as  an  add-on  to  the  existing  full-scale-development  and  production 
contracts  in  place  for  the  F-16A/B.  Those  contracts  provided  the  basis  for  Stage  I  of  MSIP 
and  preliminary  work  on  Stage  II.  Stage  II  marked  a  significant  *jump,”  howevmr,  firom  the 
F-16A/B  to  the  F-lfiCVD.  To  accommodate  that  jump,  the  F-16  SPO  and  General  Dynamics 
negotiated  a  new  development  contract.  The  contract  defined  the  terms  under  which  MSIP- 
related  design  changes  were  developed  and  translated  into  engineering  change  proposals 
that  could  be  implemented  in  the  standing  multiyear  production  contracts  for  the  F-16.  It 
facilitated  the  development  of  Blodts  30  and  40  for  the  F-16C/D.  The  F-16  SPO  and  General 
Dynamics  used  it  also  to  initiate  Block  60,  but  ultimately  returned  to  the  negotiating  table 
and  initiated  a  new  development  contract  to  continue  MSIP.  This  subsection  explains  these 
evolutionary  events  in  more  detail. 


Table  6.4 

Key  Contracts  in  F-16  XfSlP 


Contract 

Brief  Summary 

F83667-75-&0310 

FPI 

F-16  FSD  contract  provided  basw  for  initial  devtiopment  work  on 
Stages  I  and  II 

F3S667-7S47-0669 

FPI 

F-16  production  contract  paid  for  ECPs  in  Stage  I 

F38667-82-C-2038 

FPI 

MSIP  ctmtract  evolved  from  changes  in  F-16  FSD  contract;  provides 
new  basis  for  erqiedited  dianges  to  effect  Stages  II  and  m. 

F33667.68.C6009 

FPI 

MSIP  foUow-on  evolved  from  changes  in  MSIP  contract  to  provide 
basis  for  Stage  m — ^Block  60  dsv^pment 

-56- 


Stag*l 

The  F-16  SPO  authorized  General  Dynamics  to  proceed  with  Stage  I  long-lead 
arrangements  in  February  1980,  for  production  delivery  in  May  1982,  by  preparing  an 
engineering-change  order,  ECP  0350.  The  ECP  added  tasks  to  the  eiritttjng  full-scale- 
development  and  production  contracts.^^  Over  1980,  the  Air  Force  added  tasks  to  this  ECP. 
By  the  end  of  the  year,  it  had  a  not-to-exceed  value  of  $68.2  million.  Continuing  a4ju8tments 
in  the  to  be  included  delayed  definitization  until  February  1982.  The  ctunulative  value 
(target  price)  of  Stage  I  tasks  at  that  time  was  $104.6  million,  which  included  both 
development  and  production  tasks.  Modified  aircraft  were  already  completing  production  by 
this  time.  The  first  was  delivered  in  November  1981. 

Both  contracts  used  to  implement  Stage  I  were  fixed-price-incentive  firm  contracts. 
The  full-scale-development  contract  had  a  90/10  share  line,  making  it  dose  to  a  cost-plus 
contract  up  to  its  ceiling  at  130  percent  of  the  target  price.  The  production  contract  had  a 
much  more  moderate  share  line  and  lower  ceiling  relative  to  its  target  price. 

Stages  II  and  III 

The  F-16  SPO  had  developed  an  acquisition  plan  for  Stage  n  by  early  1981.  The  plan 
would  bring  new  tasks  into  the  program  as  individual  contract-change  proposals.  To 
facilitate  its  implementation.  General  Dynamics  asked  that  changes  be  consolidated  to 
indude  tasks  with  similar  technical-design  maturities  and  related  production-incorporation 
dates.  General  Dynamics  submitted  its  first  contract-change  proposal,  CCP  9101,  under  this 
new  ^stem,  in  February  1981.  It  called  for  testing  and  integration  of  core  avionics,  the 
radar  altimeter,  the  LANTiHN  HUD,  and  what  would  become  the  AFG-68  fire-control  radar. 
The  F-16  SPO  gave  General  Dynamics  until  July  to  submit  a  firm  Air  Vehide  Specification 
and  firm-price  proposal.  This  proposed  change  to  existing  contracts  marked  the  first  step 
both  beyond  Stage  I  and  into  development  of  the  F-16C/D  itself. 

Contract-change  proposals  cumulated  quickly.  Genmal  Dynamics  had  submitted  a 
second  proposed  change,  CCP  9103,  concerning  the  avionics  intermediate  shop,  the 
maintenance  activity  that  would  be  required  to  support  the  F-16C/D  in  the  field.  By 
December  1981,  the  F-16  SPO  had  authorized  $219.2  million  for  MSIP II  contract-change 
proposals.  Meanwhile,  other  change  proposals,  developed  to  integrate  new  subsystems  into 
the  F-16  brfore  MSIP  became  a  formal  program,  continued.  They  would  provide  input  for  the 
MSIP  program  in  the  future. 

l^It  was  incorporated  in  fuD-scale-development  contract  F33657-75-C-0310  by  trudification 
F0138S  and  in  production  contract  F3S657-78-C4)6^  by  modification  P00128. 
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CCP  9101  led  to  the  initiation  o£  a  fwmal  MSIP  contract,  F33657-82-C-2038,  in  June 
1982,  establishing  the  basic  t«nna  and  ccovditiona  for  all  MSIPn  change  proposals.  It 
efifectivefy  provided  a  contracting  environment  in  which  to  add  new  tasks  as  they  arose.  Hie 
contract  provided  a  pre>negotiated  fixed-price-incentive  firm  arrangement  with  an  80/20 
share  line,  a  ceiling  125  percent  above  target  price,  and  a  12>pacent  profit  rate  for  new 
contract-change  proposals.  It  established  a  Fast  Action  Negotiation  Group  (FANG) 
procedure  to  Jefinitire  quickly  all  contract  changes  under  $10  million.  The  umbrella  contract 
ipclujaj  oorreeti<ai  of  defidendes  and  econmnic  price-adjustment  (EPA)  clauses  and  provided 
for  fiezible  progress  payments.  CCP  9101  effectively  became  the  first  of  a  "OOOO  aeries”  of 
MSIP  n  and  III  changes  definitized  under  this  new  contract. 

By  1983,  the  MSIP  contract  had  become  one  of  the  six  largest  contracts  administered 
by  the  F-16  SPO.  Table  5.5  charts  its  evolution  through  the  19808.  From  an  initial  target 
price  of  $144.0  millinn,  it  grew  progressively  to  well  over  $1.3  billion  by  1988.  Subsequent 
budget  cuts  and  a  new  contract  (see  below)  led  to  adjustments  that  eliminated  tasks  from 
thig  MSIP  contract,  leaving  it  with  a  (target  price)  value  of  $1  billion  by  the  end  of  1990.  The 
substantial  growth  and  subsequent  contraction  of  the  contract’s  value  help  illustrate  the 
flexibility  allowed  by  the  contract.  Because  this  contract  was  considered  about  90  percent 
complete  by  the  dose  of  1990,  $1  billion  is  a  reasonable  estimate  of  the  full  value  of  the 
contract  over  its  lifetime. 

Stage  m  began  in  the  course  of  this  formal  MSIP  contract  CCP  9226  to  the  MSIP 
contract  initiatiiJ  the  basic  program  for  Blodc  40  of  the  F-16C/D  development  program.  It 
authorized,  in  July  1985,  the  studies  and  initial  long-lead  software  development  and 


Table  6.5 

Value  of  the  MmP  Contract  over  Hme 


Year 

Target  Price 

Effective  Ceiling 
Percentage 

Expected  Value  yVhen 
Contract  Is  Conq>lste 
Contractor  SPO 

($  million)  ($  million) 

1983 

407.5 

1.287 

453.8 

611.4 

1984 

500.4 

488.4 

1985 

625.2 

1.114 

623.9 

622.0 

1986 

843.6 

1.114 

838.1 

833.1 

1987 

992.8 

1.116 

985.1 

990.3 

1988 

1366.4 

1.073 

1367.0 

1372.6 

1989 

1021.6 

1.114 

1038.0 

1037.3 

1990 

1006.5 

1.104 

1030.5 

1025.5 

SOUBCE:  UA.  Air  Force,  Aemunticel  Sretema  Omnon,  F-IS  SPO,  SeUettd 
Aequuition  Rtportt  for  31  December  of  the  reare  ehomi. 
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integration  efforts  in  Block  40.  Full  authorization  fdlowed  in  April  1M6.  It  was  definitized 
in  early  1987  for  $296  million.  Table  5.6,  which  presents  the  conhraet-diange  proposals 
associated  with  the  Block  40  program,  illustrates  dearly  how  MSIP  divided  blodk  programs 
into  famkw  assodated  with  individual  subssmtems.  Each  of  these  tasks  ultimately  led  to 
corresponding  engineering-change  proposals  that  were  used  to  modify  Multiyear  Production 
Contract  11,  in  effect  for  FYs  86-89.  Hence,  the  tasks  shown  here  and  resources  associated 
with  them  are  primarily  development  tasks. 

Block  50  of  Stage  III  began  in  a  similar  manner.  CCP  9903  to  tiie  MSIP  contract,  a 
preliminary  Hadgn  effort  for  Block  50  aircraft  with  initial  production  deliveiy  in  June  1991, 
received  initial  authorization  in  September  1986  and  full  authorization  in  January  1987. 

CCP  9903  would  authorize  full-scale  integration  efforts.  CCP  9904  would  authorize  long-lead 


Table  &6 

Contract-Change  PnqiMaalB  on  MSIP  Contract  for  Blodc  40 


CCP  Number 

CCP  Subject 

Authorization 

Date 

9211R1 

Advanced  IFF  6rm  proposal 

Dec  87” 

9216C4 

LANTIRN  flight  test  support 

Feb  87* 

9226R1 

Basic  program  for  Block  40 

Jan  86* 

9226R1C1 

Clarification  dbasic  Block  40  program 

A4>r  86 

9226R1C2 

Additional  changes  within  NTE  of  basic  Block  40 

j^r86 

9226R1C3 

Additional  changes  and  redirection  within  NTE 

Sep  86 

9226R1C4 

AN/AFG-68  radar 

diupproved 

9226R1C5 

Suppm^  equipment  schedule 

disapproved 

9226R1C6 

Instrtscaentasion  changes 

Jul86 

9226R1C7 

Radio-frmiuency  notch  filter 

Sep  86 

9226R1C8 

Increased-capadty  battery 

Sep  86 

9226R1C9 

LANTIRN  navigation  pod  BPS 

Sep  86 

9226R1C10 

Electronic  warfisre  bus  requirements 

Sep  86 

9226R1C11 

Suppwt  equipment  sebed^ 

Jun86 

9226R1C12 

Electronic  countermeasures  pod  interfiaoe 

Sep  86 

9226R1C13 

Aft-seat  HUD  monitor 

Sep  86 

9226R1C14 

ALE-40  and  AVTR 

Sep  86 

9226R1C16 

AN/APG-68M  radar 

Sep  86 

9226R1C16 

Late  government-furniahed-eqnipment  impacts 

Jun  88* 

9226R1C17 

Special  project 

7 

9226R1C18 

IDR2move 

Mar  87* 

9226R1C19 

Additional  items 

Feb  87* 

9226R1C20 

SERD  75601  quantity 

May  87* 

9226R1C21 

*No  SOL”  sdi^ule  impacts 

Jun  87* 

9227 

Avionics  intermediate  shop  impacts 

Aug  86 

9227R1 

Avionics  intermediate  shop  impacts  update 

Aug  87* 

9236 

Condaet  additional  LANTIRN  fli^t  tests 

Mar  87 

9236 

Yuma  Proving  Grounds  omqtatibility  with  test  F-16Cs 

May  87 

9236C1 

Additional  GPS  fU^t  test 

Feb  88* 

SOURCE:  F-16  Program  Uaiugimtnt  Plan,  pp.  H-18-H-21. 

NOTE:  All  suthoriutiaB  datM  an  actual  unleM  aoAai  by  an  aatoriak  (*),  whidi  indicatM 
dataa  aw  planned. 
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activities  in  early  1987.  These  ^orts  led  to  a  new  etmtract,  F3S667-89-0009,  signed  in 
December  1988  and  definitized  in  September  1990.  It  is  a  fized-prioe-incentive  firm  ccmtract 
with  terms  similar  to  its  predecessor,  -82-C-20S8.  And  like  the  contracts  before  it,  it  has 
grown  in  value  over  time.  With  an  initial  target  price  of  $76.4  milHan,  it  had  grown  to  a 
target  price  of  $162.7  million  by  the  dose  of  1990,  with  a  ceiling  price  only  6.8  percmit  higher. 
We  can  expect  additional  growth  as  the  MSIP  program  continues  to  add  tasks  to  this 
contract.  like  its  predecessor,  this  contract  uses  contract-diange  proposals  that  lead  to 
engineering-cfaange  proposals  in  the  current  multiyear  production  contract 

Discussion 

Taken  together,  these  contracts  indicate  that,  throu^  1990,  the  Air  Force  had  agreed 
to  pay  General  Dynamics  about  $1.3  billion  in  then-jrear  dollars  for  developmmit  activities 
associated  with  MSIP.  Discounting  these  obUgations  to  1980  dollars  yields  a  base-year  cost 
of  about  $980  million.  (Because  expenditures  follow  obligations  in  time,  the  base-year  value 
of  expenditures  would  be  somewhat  lower.^^)  This  figure  does  not  indude  the  costs  of 
int^puting  the  LANTIRN  system  and  GPS,  which  are  covered  by  the  budgets  of  other 
programs,  or  the  government’s  own  costs  for  test  facilities,  assets,  activities,  and  general 
administration.  On  the  other  hand,  we  cannot  allocate  all  of  these  costs  to  the  development 
of  one  F-16C/D  design.  MSIP  has  conducted  development  activities  for  more  than  one  lype  of 
F-16CVD  and  has  also  created  the  potential  for  improving  the  capabilities  of  Block  15  F- 
116A/Bs.  As  noted  in  Section  4,  the  program  has  also  facilitated  a  series  of  odier  aircraft 
development  activities  in  the  F-16  SPO. 

Viewed  in  this  way,  these  contracts  have  successfully  handled  a  rapidly  evolving 
program  and  one  that  experienced  considerable  change  within  each  block.  The  key  to  this 
success  has  been  the  decision  to  design  the  contracts  efifoctively  as  contractual  environments 
that  enable  rapid  contractual  revision  as  new  infiirmation  becranes  available.  Within  these 
envmmments,  the  share  lines  distribute  a  substantial  share  of  risk  to  the  govmnment.  But 
moderate  ceiling  ratios  have  limited  the  government’s  exposure  and  have  not  been  exercised 
in  practice.  These  development  contracts  are  badmd  up  by  provisions  in  the  inoducti<m 
contracts  that  give  General  Dynamics  dear  responsibility  fin*  integrated  tystem  performance. 
The  potential  obvioudy  exists  for  abuse  oftheflexitnlity  that  sudi  contracts  provide.  For 
example,  definitization  of  individoal  contract-change  (»ders  has  often  talran  longer  than  one 

diacoantiisiiig  factors  inferred  from  the  F-16  SefectedAcQiiMtUowftport,  31  December 
1960.  Aasunimg  that  average  expenditure  occurs  two  to  three  years  following  an  obli^tim  and  that 
the  average  discount  rate  is  6  percent,  the  base-year  value  d’er^enditures  would  be  about  10  to  15 
percent  lower. 
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mil^  eoqMct  in  a  standard  amtnet  But  the  histoiy  of  the  program  <rfEn:8  little  evidence 
tiiat  sudh  abuse  has  occurred.  Quite  the  contrary,  the  flezilality  tiered  by  the  contracts  has 
probaUy  contributed  substantially  to  the  devetopers’  ability  to  respond  creatively  and 
eflectivdy  to  unexpected  eventa 

SUMMARY 

MSIP  has  introduced  a  aeries  of  technological  improvements  to  the  F-16  aircraft, 
moving  the  program  beyond  the  F-ISA/B  to  the  F-16C/D  and  increasing  the  capability 
of  the  F'lStyO  over  time.  MSIP  has  used  a  series  of  miniUodES  to  introduce  improvements. 
This  approach  allows  the  F-16  to  exploit  the  latest  technology  available  and  to  resolve 
incrementally  risks  associated  with  the  new  capabilities,  so  that  risk  management  for 
new-technology  introduction  builds  on  the  fimtingw  of  earlier  test  activities.  The  effort  had 
cost  about  $1  billum  throu^  1990. 

Sudi  a  program  requires  coordination  of  a  large  number  of  parallel  development  and 
integration  efforts.  The  simultaneity  of  those  efforts  creates  the  risks  of  greatest  concern  to 
MSIP  managers.  The  stability  of  the  F-16  program  as  a  whole,  reflected  in  a  continuing 
series  of  multiyear  production  contracts  over  the  course  of  MSIP,  enabled  MSIP  managers  to 
focus  their  concerns  about  risk  on  the  development  program  itself.  By  keeping  all  parties 
affected  by  the  development  activdy  involved,  assigning  the  U.S.  Air  Force  priority  in  the 
design  of  new  configurations,  seeking  derivative  subsystem  developments  to  coordinate  with 
the  MSIP  effort,  getting  involved  in  subsystem  development  programs  early  and  integrating 
them  with  an  iterative  testing  procedure,  using  progressive  introduction  of  new  capabilities 
to  limit  simultaneity,  and  carefully  managing  the  joint  use  of  test  resources  and  data,  MSIP 
has  satisfimtorily  managed  the  development-related  risks  associated  with  the  program. 

Maintaining  a  hi^y  flexible  management  environment  has  been  key  to  the  MSIP 
approadi.  The  contracts  used  to  implement  MSIP  are  essentially  contracting  environments 
that  allow  otmtinual  adjustment  as  infimnation  accumulates  in  the  development  effort  The 
fixed-price-inoentive  contracts  shift  much  of  the  risk  associated  with  such  flexibility  toward 
the  Air  Force  for  small,  unexpected  increases  in  cost  But  thqy  also  cap  the  Air  Force’s  risk 
with  moderate  ceiling  ratios  and  a  requiranent  that  General  Dynamics  retain  integrated 
syatem  performance  responsibility  for  the  F-16  weapon  system  that  results  finm  MSIP 
efforts. 
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6.  C0NCLU8I0IIS 


The  F-16  Muhinatkm*]  Staged  Improvement  I^rogram  (MSIP)  providee  a  usdiil 
example  of  a  derivative  aiicraft  development  program  that  Ikaaeesentially  added  capabilities 
dev<d<qied  daewhere,  in  the  form  of  modnlar  subqrsteme,  to  an  F-16  hane1ine  qmtmn  to  create 
the  F-16C/D.  Such  an  approadi  afSen  one  potential  way  to  respond  to  tiie  rapidly  changing 
Hafiwiaa  environment  we  face  today,  llie  threat  is  no  hmger  as  wdl  defined  as  it  was  in  the 
past  We  can  expect  it  to  change  quickly  over  time  as  new  mtemational  amtingenciee 
develop,  requiring  the  Air  Farce  to  refine  its  weapons  and  tactics  in  re^Kmae.  Wecanalso 
expect  that  the  Air  Force  will  have  fewer  real  resources  with  vritich  to  respond.  MSIP  offers 
a  way  to  update  a  proven  weapcm  system  with  apectfic  new  capabilities  as  technological 
advance  makes  them  available  or  as  the  threat  makes  them  more  important  to  the  Air  Force. 
As  we  review  the  mejm:  findings  of  this  study,  let  us  also  consider  what  thqy  tell  us  about 
potential  results  of  applying  an  approach  similar  to  MSIP  in  other  areas  of  Air  Force 
acquisition. 

IffilP  la  more  aniqn^oneh  than  a  fionaal  pragraaa.  BfSIP  offiBrs  a  way  to  think 
about  derivative  development.  Ihe  F-16  SPO  has  used  methods  like  those  in  MSIP  to 
upgrade  its  F-16A/B  fleet,  to  plan  for  successors  to  the  F-16G/D,  and  to  design  variations  on 
the  F-16  that  could  meet  specific  new  requirements.  One  variation  emulates  Soviet  aircraft 
for  the  Navy.  Another  provides  new  reconnaissance  capabilities.  A  third  enhances  air 
defense  capability  for  the  Air  National  Guard.  A  feurth,  still  under  consideratitHi,  could 
provide  enhanced  dose  air  support  Each  variatam  was  or  could  be  designed  and 
implemented  quiAly  as  new  requirements  were  recogniied.  Development  costs  have  been 
wdl  below  those  associated  with  nevHy  designed  aircraft.  Sudi  a  development  c^>ability 
should  be  attractive  in  an  environment  with  an  uncertain  threat  and  limited  resources. 

It  is  not  easy  to  integrate  modular  sobsystems.  It  is  difficult  to  fneitifjiiTi 
multiple  sabqyatems  on  the  shdf  and  integrate  them  into  an  effective  mw  qystem  whenever 
sudi  a  new  system  is  required.  MSIP  has  cost  about  llbUlkn,  “simply”  to  integrate 
capdiflitiea  devdoped  daewhere.  And  it  would  have  cost  a  great  deal  more  if  MSIP  had  not 
become  involved  ear^  in  the  devd«qiment  of  those  capabilities.  MSIP  has  taken  a  inoactive 
ap^oadi  to  identi^fing  the  new  capabilities  it  mi^  use,  establishiagrdatwoshipe  to  bring 
those  capabilities  into  a  farm  it  could  use,  and  managing  the  find  integratfam  of  maiy 
sobsystesu  into  a  coherent  qystsm.  In  the  end,  an  effective  system  must  customize  its 


subqmtems  to  fit  its  wei^t,  space,  data,  power,  display  and  ocmteid,  and  specific  system 
requirements,  and  customization  requires  time,  resources,  expertise. 

That  said,  integration  efforts  undertaken  MSIP  to  develop  the  F-16C/D  have 
significantly  eased  the  diflSeulty  of  integrating  the  same  subsystems  or  oonstellati<m8  of 
subsystmns  into  other  aircraft  variatiims.  In  particular,  the  integratum  of  sidwsrstems  into 
U.S.  verritms  of  the  F-16C/D  has  substantially  reduced  the  difficulty  integrating  them  into 
fordgn  versions. 

In  the  future,  then,  the  Air  Force  should  not  try  to  respond  to  an  unomtain  threat  by 
simply  inventorying  many  subsystem  capabilities  on  the  shelf.  MSIP  suggests  that  it  is  best 
to  develop  those  subsystems  within  some  more  fi>cus6d  context  so  that  they  reflect 
integration  concerns  and  so  that  the  Air  Force  establishes  some  experience  in  integrating 
them  wiffi  other  systems  that  it  can  appfy  quickly  when  it  needs  the  subsystems  in  new 
contexts. 

A  teet-aiialyse>flz  (TAF)  approach  requires  concurrent  integration  of 
subesrstems.  Concurrent  intention  efforts  raised  MSIPs  greatest  concerns  about  risk  in 
its  planning  process,  concerns  that  were  realized  in  many  of  the  subsystems  examined  in  the 
^rpendix.  But  ooneurrent  integration  is  a  response  to  yet  another  problem.  MSIP  exhibited 
a  strong  commitment  to  a  TAF  development  approach  that  allowed  many  iterations  of 
empirical  testing  to  identify  and  resolve  problems.  Such  an  approach  is  especially  important 
when  int^rating  many  subsystems  because  even  the  best  analysis  and  modeling  cannot 
predict  the  proUems  that  will  arise  when  complex  subsystems  interact.  Empirical  testing  is 
necessary.  Iterative  testing  is  also  necessary  to  ensure  that  solutions  to  problems  associated 
with  one  interaction  do  not  create  problems  with  anoUier  interaction.  To  implement  its  TAF 
approadi,  MSIP  has  tried  to  get  involved  as  early  as  possible  in  the  development  programs 
for  its  subs3rstem8  and  b^in  looking  at  interactions  between  those  subsystems  and  some 
useful  F-16  configuration.  Such  an  approach  leads  to  heavy  concurrency  within  the 
development  process. 

An  alternative  to  this  approach  presumably  uses  iterative  testing  to  int^rate  one 
qmtem  at  a  tinm.  The  flexible  response  needed  to  respond  most  effectively  to  problems 
discovered  in  such  a  process  requires  that  earlier  integrations  continually  be  reopened  and 
atyusted.  Such  an  approach  introduces  its  own  risks  and  significantly  increases  the  length  of 
time  required  to  adueve  a  complete  system  integration.  One  of  tlm  attractions  of  MSIP  for 
future  application  is  its  ability  to  implement  derivative  designs  quickly.  It  depends  on 
concurrency  to  offer  this  benefit.  Future  users  of  an  approach  like  MSIP  should  be  prepared 
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to  aoeqyt  the  riaks  created  bgr  eoneurrangr;  they  should  also  )dan  finr  them.  Weaaymoare 
about  oooeiirreiiqr  below. 

Lrtrodaetkm  of  new  eapabiUties  in  stages  aasidianitea  ridu  eauaad  by 
ooncurreney.  The  desirability  d  ctmeumncy  is  dmouaty  not  abedute.  MSIP  justifies  its 
staged  approach  as  a  way  to  introduce  new  subqrstan  capabilities  as  they  become  availatde 
or  attractive  and  as  a  way  to  limit  the  costs  of  retrofitting  sudi  capabilities  by  anticipating 
them  and  making  provision  for  future  subsystem  incOTporattens  on  the  inroductum  line.  The 
approach  also  helps  ameliorate  risks  associated  with  concorreniy.  In  laectiee,  MSIP  has  not 
found  it  necessary  to  reopen  many  decisions  made  in  earlier  stages  when  it  reached  later 
ones.  So  it  has  found  a  way  to  raise  a  limited  set  of  integration  risks  at  a  time— 4n  one  stage 
or  block— resdve  them,  then  raise  another  set,  resolve  them,  and  so  on.  The  delay  imposed 
by  such  a  staged  approach  is  o£EMt  by  the  foct  that  interim  stages  actually  produce  aircraft 
with  enhanced  capabilities.  Such  technologieal  capabiUttes — presumably  those  whose  risks 
can  be  resdved  quickly— become  available  early;  others  with  more  comidex  integratiem 
problems  become  availaUe  later. 

Future  developers  could  use  a  similar  staged  iq>proacb  to  limit  some  of  the  risk 
associated  with  concurrent  subsystem  integrations  and  to  speed  the  introduction  of  new 
capabilities  that  pose  fewer  integration  risks.  Such  a  staged  approach  must,  of  course, 
consider  the  cost  of  retrofitting  capabilities  developed  late  into  Qrstems  produced  early  and 
the  operating  and  support  costs  of  managing  multiple  variations  of  a  s]rBtem  if  retrofits  do 
not  ensure  uniformity  when  the  development  is  complete. 

Hie  cost  savings  govenunent-ftmiialied  equipment  may  be  overstated 
during  develcqnaent.  MSIP  preferred  developing  most  mqjor  subqmtems  as  government- 
furnished  equipment  to  save  mon^.  The  F-16A/B  had  converted  many  subsystems  to 
government-furnished  equipment  firom  contractor-furnished  eipiipment  to  save  mcmey. 

MSIP  continued  that  approach.  There  is  no  doubt  that  treating  a  ntepn*  subsystem  as 
govemment-fhmiahed  equipment  during production,  when  its  design  is  fairly  stable,  can 
save  the  Air  Force  substantiai  amemnts  of  mcmey  by  avoiding  certain  payments  to  the  prime 
contracter.  Hawevr,  maintnining  a  nf  dufirg 

efeoefopment  msy  complicate  integratkm  efibrts.  If  it  does,  resulting  shortages  of  subssrstem 
prototypes  to  be  used  as  test  assets  for  other  subsystems  and  delays  in  upgrading  these 
prototypes  can  significantly  reduce  the  effectiveness  of  integratiem  testing.  Loss  of 
effectiveness  raisM  the  cost  of  development  in  ways  that  a  standard  accounting  system 
cannot  easily  capture:  Subsystem  assets  standing  idle  for  lack  of  adequate  test  assets  can 
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impoM  sabstantial  opportunity  coots  tiiot  mn  now  irttributod  to  the  "»«— »"g  toot  ooooto.^  If 
maintaining  a  oob^stem  as  gownmant-fumiahed  equipment  contributes  to  test-aooet 
sluntage,  doing  so  imposes  costs  that  cannot  easily  be  nmasured,  but  tb^  are  costs  that 
could  euily  cdEiet  the  apparent  cost  savings  associated  with  govemnmnt-fumidied 
equiimient 

\f^tliont  a  more  detailed  analysis  (tf^BfSIP,  we  cannot  say  how  impmrtant  this  problem 
has  been.  Surely,  the  Air  Force  required  General  Dynamics  to  assume  integrated-system- 
performance  responsibility  for  its  key  govemment-fumished  subsystems  to  ameliorate  the 
int^pntkm  problems  that  mi(^t  arise  firtnu  maintaining  those  subqrstems  as  government' 
furnished  equipment.  We  cannot  say  how  successful  that  gambit  has  been.  Future 
developers  contemplating  an  approadi  like  MSIP  should  consider  carefully  the  costs  and 
benefits  of  govemment-fumished  equipment,  recognizing  that  standard  cost  accounts  cannot 
provide  the  information  needed  to  make  the  ri^t  decision.  They  should  give  closest 
attention  to  those  mqjor  subsystems  that  will  require  the  greatest  integration  efiiat. 

The  development  riaka  presented  by  MSIP  call  for  a  flesiUe  management  and 
contraetiiig  structure.  The  types  of  risks  that  MSIP  planners  associated  with  concurrent 
integrations  are  precisely  th<we  that  cannot  be  resolved  at  the  beginning  of  a  development  in 
a  management  plan  or  by  contractual  arrangements.  Although  the  planners  knew  that  many 
surprises  would  arise,  thqy  could  not  know  where  they  would  arise  or  when,  or  what  th^ 
woiild  be.  Any  attempt  to  set  up  explicit  arrangements  for  such  surprises  in  advance  would 
yield  a  fairly  blunt,  wooden  development  environment  that  could  not  respmd  well  to  the 
specific  surprises  that  actually  arose.  This  statement  could  probably  be  made  about  any 
development  effort  expecting  to  resolve  significant  uncertainties  over  its  course;  it  is 
especially  true  of  developments  like  MSIP,  the  risks  for  which  arise  from  many  integrations 
whose  interactions  simply  caimot  be  foreseen. 

Such  a  situation  calls  tor  a  "relational  contract”  among  the  parties  involved,  a 
contracting  environment  that  strives  harder  at  Aafining  and  iwaiTifaiiniTig  relationships 
among  the  parties  than  it  does  at  resolving  specific  difficulties  in  advance.^  Such  an 
arrangement  relies  heavily  on  custom  and  historical  rriationships  among  the  parties,  but 
fcnnal  instruments  can  enhance  it.  The  contracts,  memoranda  of  agreement  and 
understanding,  working  groups,  and  other  management  arrangements  associated  with  MSIP 
exempli^  the  types  of  instiuments  required,  the  finmats  in  which  specific  tasks  can  be 

iPor  a  lucid  and  entertaining  description  of  this  problem  in  a  production  setting,  see  Goldratt 
and  Cox,  Idgfi. 

SFor  an  dabaratkm  <iS this  idea,  see  MacCaulay,  1968;  Macneil.  1980;  Williamson,  1979, 1985. 
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structured  as  the  need  arises.  Such  instruments  impose  dose  otmtrol  on  the  tasks,  if 
necessary,  but  maintain  the  flexibility  to  address  new  tasks  in  the  best  way  possible  for  those 
involved. 

Althou^  disputes  betwemi  oi|puiizations  inevitably  arose  during  MSIP  and 
responsilnlities  were  occasionally  reassigned  to  deal  with  such  problems,  MSIP  has  been 
remarkably  firee  of  fundamental  disagreements  among  the  parties  involved— all  the  more 
remmrkable  given  the  number  of  parties  involved  and  the  turbulmee  in  the  (tevelopments  Ah: 
many  of  the  subsystems  they  represented.  Future  developers  interested  in  using  an 
approach  like  MSIP  would  benefit  significantly  fiom  looking  more  doeely  at  the  specific 
arrangements  established  for  sustaining  organisational  relationships. 

Incentive  contracts  treat  a  limited  set  of  risks.  Much  of  the  literature  on  defense 
contracting  focuses  on  whether  a  contract  is  cost  based  or  fixed  price,  on  the  share  line  that 
allnrafam  risk  between  the  buyer  and  supplier,  and  on  the  price  ceiling  that  limits  the  buyer’s 
risk.  Althou|d>  these  factors  are  important,  address  only  part  of  the  contracting 
problem,  the  specific  set  of  risks  assodated  with  specific  tasks’  accounting  costs.  They  do  not 
deal  with  the  nugor  risks  encountered  in  MSIP. 

MSn*  planners  expected  integration  concurrency  to  contribute  the  largest  risk  to 
MSIP.  When  shortages  of  updated  prototypes  complicated  MSIP,  imposing  costs  not  just  on 
MSIP  but  on  maity  of  the  devdopment  programs  feeding  into  MSEP,  the  fixed-price-incentive 
MSIP  contract  in  place  could  do  nothing,  by  itself,  about  the  costs  that  the  Air  Force  (and  its 
contractors)  bore  as  a  result.  Among  the  histmies  presented  in  the  ^pendix,  the  APCt-68 
provides  the  most  compelling  illustration  of  this  problem.  After  exceeding  its  price  ceiling, 
Westin^ionse  bore  the  fuU  burden  of  the  costs  it  assodated  with  developing  the  APG-68.  But 
its  problems  continued  to  impose  substantial  costs  on  the  Air  Force  and  other  contractors 
that  the  incentive  contract  aid  not  even  address.  The  Air  Force  could  use  Westinghouse’s 
failure  to  perform  as  leverage  for  concessions  elsewhere.  However,  the  form  such  leverage 
would  take  could  not  be  predicted  or  even  addressed  in  terms  of  the  share  line  and  ceilings  so 
often  emphasised  in  discuMions  of  contracting.  Other  terms  of  the  contract,  many  of  them 
implicit,  were  more  important. 

More  generally,  MSIP  has  offered  a  context  in  which  the  work  statement  is  contint'ally 
beingrevised.  AHhou^  specific  price  terms  apply  to  each  new  task  negotiated,  tlw 
continning  flexibility  of  MSIP  offers  ample  iqiportunity  to  override  or  ren^jotiate  those 
terms.  That  more  general  envinmment  is  more  important  to  understanding  MSIP  than  the 
specific  terms  of  any  one  task.  We  fimnd  no  evidence  that  the  parties  to  IdSIP  have  abused 
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the  flezibiUty  allowed  by  the  program.  But  future  users  of  the  approach  should  be  aware  of 
the  potential  for  abuse. 

Good  managers  made  MSIPs  flexible  environment  woric.  MSIP  provides  its 
managers  with  considerable  freedom  to  act;  it  can  do  so  only  because  the  MSIP  plaimers  had 
confidence  that  MSIP  managers  would  act  properly  and  effectively.  MSIP  provides  a  flexible 
environment  in  which  developers  can  react  to  surprises  as  they  arise.  Given  the  nature  of 
risks  in  the  F-16  development,  the  contracting  literature  would  generally  support  such  an 
approach.  That  literature  gives  little  attention  to  the  question  of  how  developers  will  exploit 
that  flexible  environment. 

In  MSIP,  management  experience  does  not  necessarily  lie  in  the  SPO  leadership. 
Although  the  F-16  SPO  was  well  established  and  organized  well  before  MSIP  began,  MSIP 
has  been  run  primarily  by  military  personnel  with  limited  tenures  at  the  SPO.  That  limits 
their  accumulation  of  experience  working  in  the  MSIP  environment.  Greater  stability  has 
existed  in  the  predominantly  civilian  functional  offices  of  the  SPO  and  at  General  Dynamics. 

A  program  like  MSIP,  which  constantly  presents  opportunities  to  reopen  the 
relationship  between  the  Air  Force  and  the  prime  contractor,  offers  the  potential  for  sldlled 
personnel  to  exploit  those  opportunities  in  specific  circumstances,  endangering  the  broader 
relationship  over  the  long  run.  That  has  not  happened  in  MSIP.  Finding  the  reason 
deserves  more  attention.  Perhaps  the  staffs  of  the  SPO  and  contractor  are  well  enough 
matched  in  their  skillH  and  experienre  that  neither  has  been  able  systematically  to  exploit 
the  other.  Perhaps  they  share  a  set  of  values  that  limit  their  willingness  to  pursue 
opportunities  for  shoart-term  gain. 

Developers  considering  a  program  like  MSIP  in  the  future  must  focus  particularly  on 
the  people  who  wiU  manage  it.  Given  ...e  freedom  that  such  a  program  allows,  those  people 
will  have  more  effect  on  its  success  than  they  would  in  a  traditional  development.  As  noted 
above,  developers  should  examine  their  people’s  experience,  their  skills  and,  to  the  full  extent 
possible,  their  commitment  to  preserving  the  contracting  environment  over  the  long  term. 

Multiyear  production  contracts  have  limited  risk  in  MSIP.  For  the  duration  of 
most  of  MSir,  one  of  three  multiyear  contracts  was  in  place  for  the  F-16.  Those  contracts 
were  important  to  MSIP  for  two  reasons.  First,  they  alleviated  concern  that  the  survival  of 
the  F-16  program  itself  mi^t  be  at  risk.  Their  existence,  in  itself,  probably  only  had  a 
limited  effect  on  risk  in  MSIP.  But  the  fact  that  both  Congress  and  the  Air  Force  approved 
this  series  of  contracts  points  to  a  degree  of  stability  in  and  consensus  about  the  F-16 
program  that  had  to  ease  the  minds  of  developers.  In  particular,  such  stability  meant  that 
developers  would  not  feel  as  much  pressure  to  overpromise  to  maintain  support  for  their 


development  efforts.  Not  having  to  overpromiae  limited  their  own  risks  and  allowed  them  to 
pay  greater  attention  to  the  management  and  reduction  of  those  risks  over  time. 

The  second  reason  the  multiyear  contracts  were  important  to  MSIP  was  that  they 
mitigated  the  different  kind  of  pressure  ongoing  production  programs  put  on  developers.  As 
aircraft  were  scheduled  for  delivery,  developers  had  to  ensure  that  the  aircraft  being 
delivered  were  complete  enough  to  fly  safely.  The  developers  effectively  did  not  have  the 
option  of  simply  delaying  delivery  of  new  systems  in  response  to  a  development  problem. 
MSIP  was  deliberately  structured  so  that  if  it  could  not  deliver  a  new  subsystem  as  expected, 
placeholders  were  incorporated  for  that  subssretem  in  new  production  aircraft.  Thus,  the 
aircraft  could  operate  safdy,  but  the  new  subsystem  could  replace  the  placeholder  or  be 
retrofitted  to  achieve  the  final  capability  when  it  became  possible.  This  response  to  surprises 
during  development  occurred  repeatedly  in  MSIP. 

Developers  romridering  a  future  program  like  MSIP  should  be  aware  of  the  positive 
effects  that  the  multiyear  contracts  had  on  MSIP.  Where  such  contracts  are  used  in  the 
future,  programs  like  MSIP  may  be  more  attractive.  Where  they  are  not,  developers  should 
be  aware  that  a  development  program  like  MSIP  will  face  different  pressures:  Success  will 
be  more  difficult  to  achieve,  and  it  will  not  depend  as  heavily  on  the  strategy  that  MSIP  has 
used — ^incorporating  partially  capable  subsystems  early  and  replacing  or  retrofitting  them 
later. 

UJ5.  dominance  of  MSDP  simplified  risk  management.  Althou^  MSIP  is  known 
explicitly  as  a  mvdtinational  program  to  recognize  the  participation  of  Belgium,  Denmark, 
The  Netherlands,  and  Norway,  the  United  States  has  been  the  dominant  partner  in  this 
program  from  the  beginning.  U.S.  domiiumce  has  limited  risk  by  allowing  MSIP  to  focus  its 
activities  on  U.S.  priorities  first  and  then  to  turn  to  the  priorities  of  the  other  participants. 
That  focus  has  limited  risks  associated  with  the  development  of  specifications  for  new 
configurations  developed  under  MSIP.  It  has  also  simplified  considerably  the  integration  of 
options  offered  to  foreign  air  forces.  U.S.  dominance  has  also  reassured  other  foreign  buyers 
of  F-lSC/Ds  of  the  United  States’  continuing  interest  in  and  commitment  to  the  F-16C/D 
inngram.  This  effect  is  best  exemplified  by  the  concrnn  that  grew  among  potential  foreign 
buyers  of  the  F-16A/B  until  the  U.S.  undertook  efforts  to  upgrade  it  to  imeserve  its  utility  to 
U.S.  forces. 

Future  developers  will  probably  work  in  an  environment  in  which  multinational  sales 
become  increasin^y  important  To  promote  multinational  participation  in  a  future  program 
like  MSIP,  those  developers  mi^t  be  tempted  to  seek  a  larger  foreign  role  in  the 
development  process.  What  that  role  mi^t  be  will  obviously  depend  primarily  on  the  nature 


of  the  discussion  among  the  countries  involved.  But  U.S.  participants  in  sudi  talks  should 
keep  in  mind  that  a  dominant  U.S.  role  in  this  so-called  multinational  F-16  program  has 
contributed  significantly  to  the  success  td'MSIP,  firom  U.S.  and  foreign  perspectives.  That 
said,  the  roles  that  foreign  governments  have  played  in  the  F-16  MSIP  deserve  more 
attention  than  we  could  give  them  in  this  study. 

In  sum,  a  program  like  MSIP  offers  great  promise  for  fiiture  system  developers  who 
seek  to  react  as  quickly  and  fieribly  to  dianges  in  the  threat  as  a  derivative  development  can. 
MSn*  is  a  subtle  program  It  has  many  unique  dmracteristics  that  will  not  be  present  in 
future  efforts.  And  it  has  had  its  own  pnddems  that  future  developers  would  not  want  to 
repeat.  But  it  offers  a  good  example  of  positive  lessons  about  how  to  structure  future 
development  efforts. 


Appendix 

mSIQHTS  FROM  SIX  SUBSYSTEMS  INTEGRATED  DURING  MSIP:  AN/APG-68  HRE- 
CONTROL  RADAR,  AMRAAM,  LANTIRN,  HUD.  GPS.  AND  AFEs 

The  discussion  in  the  text  examines  MSIP  firom  above,  treating  it  as  a  sin^e 
development  program  with  unified  goals,  management,  and  schedule.  Even  viewed  firom 
above,  MSIP  reveals  itself  as  an  effort  that  coordinates  many  parallel  efforts  with  focused, 
individual  goals.  When  we  think  of  risk  as  a  situation  in  which  a  developer  mi^t  be 
surprised,  we  should  expect  sudi  surprises  to  be  easiest  to  observe  in  the  individual 
development  and  integration  efforts  comprised  in  MSIP.  This  Appendix  seeks  a  better 
understanding  of  surprises  by  looking  at  MSIP  finm  the  bottom  up. 

In  this  Appendix,  we  follow  the  integration  of  six  subs3rstems,  listed  in  Table  1.2, 
throu^di  MSIP:  APG-68  fire-control  radar,  AMRAAM  and  its  launcher,  LANTIRN  pods,  head- 
up  display  (HUD),  Global  Positioning  System  (GPS),  and  Alternate  Fighter  Engines.  Ten 
years  of  official  historical  records  for  the  F-16  SPO,  bom  1979  to  1988,  were  consulted  to 
examine  each  int^ration  and  identify  m^or  surprises,  ask  why  they  occurred,  and  ask  how 
MSIP  reacted  to  those  surprises.  These  questions  are  answered  by  addressing  each 
subsystem  in  turn.  Each  subsystem  is  described  briefly  and  a  brief  history  is  given,  then 
mqjor  surprises  are  discussed.  Table  5.1  should  provide  a  useful  temporal  context  in  which 
to  compare  timelines  for  individual  subsystem  developments,  integrations,  and  production 
incorporations  with  the  timeline  for  MSIP.  This  Appendix  closes  by  looking  across  the  six 
subsystems  and  sununarizing  the  mqjor  sources  of  surprise  and  the  principal  ways  MSIP 
managers  reacted  to  them. 

AN/APG-68  RRE  CONTROL  RADAR 
Description 

This  Westinghouse  radar  was  derived  fiom  the  Westinghouse  APG-66  radar  used  in 
the  F-16A/B.  In  fact,  until  early  1983,  it  was  kxrown  as  the  improved  APG-66.  The  key 
improvmnents  in  the  radar  were  the  addition  of  a  new  programmable  signal  processor  (PSP), 
a  dual-mode  transmitter  (DMT),  and  several  new  radar  modes.  They  enhanced  the  F-16’s 
abihfy  to  deliver  air-to-air  and  air-to-ground  munitions  in  all-weather  conditions.  By 
extending  range  beyond  visual  range  alone,  the  improvements  enabled  the  F-16  to  launch 
several  AMRAAMs  at  once,  and  they  increased  map  resolution  about  ei^tfold  in  air-to- 
ground  use  and  the  range  of  radar  modes  available.  They  also  improved  flexibility  by 
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pennitting  addition  of  new  modes  in  the  future.  These  improvements  were  expected  to  make 
the  APG-68  an  effective  alternative  to  the  Hughes  AN/APG-65  radar.^  Although  the  Air 
Force  expected  to,  and  ultimately  did,  use  a  version  of  the  improved  radar  in  the  B-IB  as  well 
as  in  the  F-16,  the  F-16  SPO  oversaw  the  development  of  this  govemment-fumished 
subqrstem.  When  the  B-1  program  selected  the  new  radar  for  its  use  in  December  1981,  the 
B-1  and  F-16  programs  set  up  a  formal  memorandum  of  agreement  and  to  develop 

plans  to  encourage  commonality  in  the  F-16  and  B-1  versions  to  be  developed. 

The  APG-66  had  been  a  contractor-furnished  itmn  in  the  F-16A/B.  The  radar’s  central 
role  in  the  F-16  aircraft  would  appear  to  favor  a  contractor-furnished  status  to  promote 
integration.  But  the  F-16  program  changed  this  status  for  the  APG-66  to  govenunent- 
fumished  to  reduce  its  acquisition  cost  to  the  Air  Force.  The  APG-68  remained  a 
govemment-fumished  subsystem  perhaps  because,  over  the  life  of  MSIP,  APG-68  contracts 
were  among  the  largest  that  the  F-16  SPO  signed.  The  APG-68  had  hi^  visibility  in  the 
SPO. 

The  development  program  for  the  APG-68  was  active  throu^out  MSIP.  Although  this 
program  was  functionally  separate  fiom  MSIP,  F-16  SPO  management  reduced  the  distance 
between  the  two  programs.  They  were  typically  managed  by  the  same  offices  within  the 
SPO.  Furthermore,  each  change  in  the  APG-68  program  precipitated  related  integration 
problems  for  MSIP.  The  presence  of  both  programs  in  the  SPO  probably  improved  its  ability 
to  manage  these  dianges  and  the  problems  that  accompanied  them. 

History 

The  development  of  this  radar  effectively  began  in  April  1980,  when  the  F-16  program 
received  direction  to  develop  a  derivative  radar  using  earlim-  development  efforts  on  the  PSP. 
The  new  design  had  the  additional  components  mentioned  above.  Its  specifications  came 
fixnn  the  common  modular  multimode  radar  program. 

Unlike  most  of  the  testing  associated  with  MSIP,  much  of  the  initial  flight  testing  for 
the  APG-68  occurred  outside  Air  Force  facilities.  The  F-16  SPO  and  Air  Force  Fli^t  Test 
Center  reached  agreement  in  late  1980  to  use  the  Westin^ouse  fli^t  test  focility  in 
Baltimore.  Westin^ouse  would  use  a  corporate-owned  Sabreliner,  modified  to  carry  F-16 
avionics,  as  the  test  platform  for  mudi  of  the  early  work.  This  approach  was  expected  to  cost 
less  than  giving  Westini^ouse  extended  access  to  an  F-16  test  aircraft  and,  incidentally, 
would  help  relieve  excess  demand  for  F-16  aircraft  test  assets. 
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Despite  the  unorthodox  test  site  and  test  vehide,  the  F-16  SPO  coordinated  thin 
development  effort  with  MSIP  early  in  its  life.  Flij^t  tests  began  on  the  Salundiner  in  early 
1982,  fbcusing  especially  on  high-risk  modes  to  assess  their  feasibility.  The  design  reviews 
that  followed  these  flints  raised  serious  omcems  about  smne  of  the  modes  tested;  developers 
expected  additional  tests  to  alleviate  those  concerns.  Sabrehner  tests  continued. 

By  early  1983,  development  was  sufBcientlly  advanced  for  WeatinehtHiae  to  sign  a 
production  contract.  The  contract  induded  a  rdiability  incentive  inrogram  in  vdiich  the  Air 
Force  would  use  subjective  criteria  to  award  Westini^iouse  a  fee  every  six  months  based  on 
the  reliability  level  achieved  in  the  new  radars.  Althou^  devdopment  was  running  a  few 
months  late,  production  schedules  were  expected  to  be  achieved.  By  this  time  Westinghouse 
also  began  development  work  on  a  version  suitable  for  fineign  military  sales. 

In  June  1983,  Westin^ouse  was  able  to  send  General  Dynamics  an  improved  APG-66, 
modified  to  interact  with  a  fiill-scale-develfqnnent  AMRAAM  tystem.  General  Dynamics 
used  this  system  to  begin  tests  in  its  Systems  Int^ration  Laboratory  (SIL).  Conflicts  in 
access  to  the  SIL  between  the  APG-68  and  LANTIUN  were  resolved  in  the  LANTIBN’s  favor, 
to  support  efforts  to  meet  its  schedule. 

As  radars  were  produced  and  delivered,  flif^t  testing  on  MSIP  F-16s  began. 
Unfortunately,  problems  delayed  the  APG-68  program;  two  restructurings  occurred  iu  1983 
alone.  Delays  in  software  upgrades  delayed  fli^t  tests  at  Edwards  and  Westm^ouse.  By 
late  1983,  a  growing  shortage  was  expected  to  affect  the  development  of  other  systems 
associated  with  the  APG-68.  To  aggravate  the  shortage,  the  radars  delivered  did  not  meet 
their  performance  specifications.  They  were  not  even  as  capable  as  the  APG-66  firom  which 
they  were  derived.  Westin^ouse  had  exceeded  the  ceiling  <m  its  fixed-price  incentive  full- 
development  contract,  effectively  shifting  the  risk  of  additional  cost  increases  solely  to 
Westin^ouse.  But  the  shortages  and  performance  diortfall  continued  to  impose  costs  on 
MSIP. 

The  F-16  SPO  responded  to  these  problems  in  a  variety  of  ways.  It  developed  an 
incmtive  program  to  encourage  more  rapid  production  of  radars.  As  the  situation 
deteriorated,  it  withheld  progress  payments  and  then  contract  prv^t  on  production 
deliveries.  The  Air  Force  paid  award  fees  of  zero  on  the  reliability  incentive  program. 

General  officers  imposed  intense  pressure  on  their  Westini^Kiuse  counterparts.  The  F-16 
SPO  negotiated  no-cost  contract-change  proposals  with  Westin^iouae  to  get  additional  work 
in  compenaation  for  the  delays.  Westmghouse  replaced  its  general  manupgr  and  developed  a 
recovery  program.  Westini^iouse  and  the  Air  Force  restructured  the  sdMdule  and  agreed  to 
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split  the  costs  c^retrofittiiig  capal^ties  not  delivered  when  expected,  to  reflect  who  had 
caused  the  delay. 

In  the  end,  Westinghouse  succeeded  in  delivering  production  APG-68s  as  agreed  for 
installation  in  the  first  MSIP  Stage  II  aircrait  in  April  1984.  But  design  of  the  DMT 
remained  incomplete  and  software  in  the  radar  could  not  meet  key  specifications.  These 
problems  persisted.  APG-88  performance  did  not  reach  the  level  of  AP6-66  performance 
until  February  1985.  Ileoovety  schedules  could  not  be  met  Software  upgrades  continued, 
finally  achieving  the  performance  specification  eq)ected  in  Octcfoer  1987.  By  that  time, 
Westinghouse  had  agreed  to  pay  for  retrofits  to  bring  previously  delivered  units  up  to  par. 
Meanwhile,  manufacturing  problems  with  the  key  new  components,  the  PSP  and  DMT, 
caused  further  problems  and  delayed  design  stability  for  the  radar.  Westinghouse  brou^t 
these  problems  under  control  by  late  1987. 

Althou^  these  aspects  of  the  APG-68  development  had  the  largest  negative  effects  on 
MSIP,  several  other  events  were  also  important.  First,  in  late  1985,  Westinghouse  began 
development  woric  to  add  VHSIC  capability  to  the  PSP,  using  funding  finm  OSD  and  the 
F-16  SPO.  Developers  expected  to  introduce  that  capability  into  MSIP  in  1991,  suggesting  a 
very  aggressive,  success-oriented  program. 

Second,  initial  tests  of  tlm  airborne  self-protection  jammer  (ASPJ)  raised  the 
possihility  of  serious  interference  with  radio  firequendes  important  to  the  coordination  of  the 
APG-68  and  AMRAAM  systems.  MSIP  began  to  address  this  possibility  in  1984.  As  tests 
accumulated,  it  appeared  that  the  problem  was  not  as  serious  as  expected.  Nonetheless,  by 
early  1986,  unavailability  of  ASPJ  assets  slowed  the  int^ration  of  the  APG-68  into  future 
MSIP  blocks.  The  ASPJ  problem  pmsisted  into  1987.  In  the  meantime,  MSIP  initiated 
efforts  to  develop  an  advanced  interference  blanking  unit  (AIBU)  and  a  radio-fiaquen<y  (RF)- 
switchable  notch  filter  to  alleviate  the  problem.  Efforts  to  achieve  RF  compatibility 
continued  into  1988  and  induded  coordination  with  the  Navy  at  the  Electromagnetic 
CompatilHlity  Analysis  Center  at  Annapolis,  Maryland;  antexma  testing  at  the  Rome  Air 
Developmmit  Center  at  Rome,  New  York;  and  anechoic  chamber  testing  of  AMRAAM  and 
other  munitiais.  In  the  end,  such  compatibility  problems  appear  to  have  been  resolved. 

Third,  in  early  1986,  OSD  recommended  that  APG-66C  radars  be  used  in  place  of 
APG-68S  in  over  half  of  the  new  F-16C/Ds  induded  in  the  FYDP.  Westin{^ouse  countered 
with  a  proposal  that  the  AFG-68  be  modified  to  reduce  its  cost  The  most  prominent  chsnge 
would  remove  fli^t-line  programmability  in  the  PSP.  The  package,  which  became  known  as 
APG-68M,  reduced  initial  acquisition  cost  and  increased  reUalnlity  to  reduce  overall 
ownership  cost  for  the  tystem.  In  August  1986,  the  F-16  SPO  received  direction  to  use  this 
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8(Hnewhat  less  capable,  but  substantially  less  costly,  veraon  of  the  radar  in  Blodc  40  aircraft 
to  be  introduced  in  December  1988. 

Finally,  the  Air  Force  sou^t  further  improvements  in  reliabilily:  an  increase  in  mean 
flight  time  between  maintenance  actions  (MFTBMA)  from  about  60  hr  fin*  the 
system  to  over  100  hr  and  ultimately  over  150  hr.  It  also  wanted  to  cut  the  initial  acquisition 
cost  of  the  radar  by  about  26  percent.  The  mw  radar  would  be  known  as  the  *three-digit” 
APG-68(V),  for  breaching  the  three-digit  MFTBMA  threshrid.  Beginning  in  early  1987, 
Westini^umse  undertook  an  Air  Force  funded  development  effort  to  adiieve  these  goals  in 
anticipation  of  providing  a  no-cost  warranty  agreement  that  rewarded  Westinghouse  for 
performances  above  100  hr.  Tim  new  radar  entered  the  F-16  on  schedule  in  December  1988. 
It  achieved  its  goal  during  the  following  year,  saving  the  Air  Force  $60  million  and  restoring 
the  mission-capable  rate  for  F-lOC/Ds  to  over  90  percent^ 

Oiscuasion 

This  very  brief  history  reveals  eig^t  surprises,  or  problems.  Speaking  broadly,  three 
involve  surprises  caused  by  technical  risk,  two  concern  surprises  associated  with  the 
availability  of  test  assets,  and  three  involve  surprises  associated  with  changes  in  system 
specifications. 

1.  Early  Indications  of  Design  Risk.  Development  managers  essentially 
maintained  their  optimism  in  the  face  of  disturbing  evidence.  Without  a  more  detailed 
examination,  we  cannot  know  whether  they  could  have  avoided  later  problems  by  taking 
these  early  indications  more  seriously. 

2.  Difficulty  Maintaining  Production  Schedules.  A  serious  problem  resisted 
repeated  efforts  to  resolve  it.  This  problem  resulted  primarily  from  an  immature  design  that 
was,  in  part,  related  to  a  series  of  produdbility  issues.  Later,  the  F-16  SPO  would  also 
attribute  part  of  the  problem  to  an  inoperienced  manufacturing  facility.  As  the  problem 
persisted,  the  SPO  brought  all  the  means  at  its  disposal  to  bear  on  resolving  the  problem.  It 
is  worth  noting  that  special  provisions  of  the  contract  provided  few  such  means.  Withholding 
progress  payments  and  profits,  bringing  hig^-level  pressure,  and  intervening  directly  in 
Westin^uraae’s  operations  would  have  been  possible  in  almost  any  contract.  Only  the 
reliability  incentive  and  fized-price-incentive  arrangements  in  the  contract  provided 
customised  instruments  for  specific  response.  It  is  also  worth  noting  that,  in  the  end,  the 
proUem  precipitated  targeted  management  interventions  that  required  the  Air  Force  to  have 
detailed  knowledge  of  WestinglMnise’s  activities. 
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T)i»  proldem  had  significant  effects  on  BfSIP  activities  that  had  not  been 
contemplated  in  any  detail  or  planned  for  in  any  contractual  device  to  whidi  MSIP  was  a 
party.  The  fact  that  the  F-16  SPO  oversaw  both  MSIP  and  the  APG<68  develt^ment 
probaUy  limited  the  damage  to  MSIP.  But  a  creative  response  firom  MSIP  managers  also 
{dayed  an  important  role. 

8.  Diffienlty  Aehiaviag  Performance  Speeificationa.  It  is  difficult  to  separate 
the  origin  of  this  problem  or  the  Air  Force  response  to  it  firom  those  of  the  one  above.  But  one 
additional  issue  conies  up  here.  Even  if  production  had  not  been  a  problem,  the  quality 
problem  would  remain.  And  MSIFs  response  to  this  problem  is  illustrative  of  a  common 
reqionse  throughout  MSIP.  MffiP  accepted  the  substandard  radars  to  maintain  its 
produetion-inoorptnatiim  schedules  and  planned  retrofits  to  correct  problems  in  these 
substandard  systems  as  solutions  cmild  be  developed.  In  a  sense,  the  substandard  radars 
served  as  placeholders  for  the  final  radar  configuration. 

This  approach  incurred  retrofit  costs,  complicated  efforts  to  integrate  the  radar  with 
other  sulMystems  while  final  corrections  were  pending,  and  reduced  the  capabilities  of  the 
aircraft  produced  with  substandard  radars.  Viewed  firom  thia  perspective,  thia  approach 
spread  the  effects  of  a  technical  surprise  over  the  cost,  schedule,  and  performance  of  the 
F-lfiCVD.  It  is  also  worth  noting  that,  in  the  end,  these  effects  were  transitional.  Later 
versions  of  the  F-16C/D  received  a  hi^y  capable  fire  control  radar,  fiilly  integrated  into  the 
aircraft. 

4.  Conflict  at  the  SIL.  General  Dynamics’  SIL  served  as  a  powerful  development, 
test,  and  int^pration  asset  during  MSIP.  It  could  not  accommodate  all  demands  placed  on  it 
by  the  various  subsystems  associated  with  MSIP.  This  is  not  to  say  that  the  SIL  should  have 
been  designed  with  greater  capacity;  without  a  detailed  analysis,  we  caimot  say  whether 
excess  dmnand  for  the  SIL  was  too  high  or  too  low  during  this  development.  When  conflict 
occurred,  however,  MSIP  managers  set  priorities  by  e»"«nining  the  relative  situations  of  the 
subsjrstems  involved  in  MSIP  and  then  gave  them  access  to  the  SIL  on  the  basis  of  those 
priorities.  No  formal  qrstem  appears  to  have  been  instituted  to  set  priorities;  MSIP 
managers  simply  wt  priorities  as  required  to  promote  the  best  use  of  the  SIL. 

6.  NoaavailaUUly  of  ASPJ  Assets.  The  persistent  nonavailability  of  ASPJ  assets 
during  MSIP  was  deariy  not  anticipated  and  was  caused  by  irresolvable  pnddems  in  the 
ASPJ  development  program.  The  Air  Force  finally  placed  the  ASPJ  program  on  hold  in 
December  1988,  pending  an  aseessment  of  its  likely  future  success.  During  MSIP,  the  Air 
Force  and  Navy  participants  in  the  joint  program  (the  Navy  was  the  lead)  had  very  different 


-76- 


views  of  how  to  manage  the  program.  We  cannot  detennine  to  what  extent  the  Navy  role  in 
this  program  compheated  MSIP  dependence  <»  this  mhaystem. 

Given  that  d^iendaiee,  however,  and  the  continuing  protdeme  that  it  created  fin- 
MSIP,  the  developers  had  Httte  specific  larateetion  |»epared  in  advance.  As  mam^ers  facing 
a  new  problem,  th^  reacted  in  several  wajrs.  Th^  began  new  devebpment  efforts  to  reduce 
the  difficulty  of  integrating  the  APG-68  and  ASPJ  system,  when  that  became  possible.  As  we 
shall  learn  in  the  LANTHtN  histoiy  below,  th^  also  considered  changes  in  the  opmition  of 
the  aircraft  and  alternatives  to  the  ASPJ  qrstem  itself. 

6.  Introduction  of  VHSIC  Capability.  Altbou^  a ‘surprise,”  this  was  less  a 
problem  then  a  simple  change  in  MSIP  plans.  MSIP  was  structured  explicitly  to  allow  such 
changpm,  even  if  this  specific  change  could  not  be  anticipated.  The  procedures  in  place 
handled  it  without  difficulty. 

7.  Introduction  of  the  Leas  Capable  APG-d6C.  This  is  an  unusual  example  of 
direct  OSD  intervention  in  MSIP.  Rather  than  accept  the  initial  proposal,  Westin^ouse 
offered  an  alternative  way  to  cut  cost,  one  that  required  far  less  sacrifice  in  performance. 
Once  this  alternative  was  accepted,  the  change  in  configuration  inevitably  required  new 
integration  tasks.  MSIP  managers,  working  within  the  standard  MSIP  process,  were  able  to 
accommodate  it  without  difficulty. 

8.  Introduction  of  the  More  Reliable  APG-68(V).  This  intervention  appears  to 
have  originated  in  the  Tactical  Air  Command,  where  low  radar  reliability  was  retarding  the 
readiness  of  the  F-16C/D  fleet.  like  the  introductions  above,  it  seemed  to  proceed  without 
serious  difficulty  in  MSIP.  The  incentive  structure  used  to  implement  it  was  unusual. 
Althou^  implemented  outside  MSIP,  it  presumably  affected  Westingfaouse’s  performance 
within  MSIP  in  a  positive  and  ultimately  successful  way. 

ADVANCED  MEDIUM  RANGE  AIR-TO-AIR  MISSILE  (AMRAAM) 

Daacriptlon 

The  AIM-120A  AMRAAM  provides  an  air-to-air  attack  capability  beyond  visual  range. 
The  Air  Force  and  Navy  developed  it  jointly  (the  Air  Force  had  lead  responsibility)  to  replace 
the  AIM-7M  Sparrow,  a  less  caq>alde,  significantly  heavier  air-to-air  missile.  When  used  with 
the  APG-68  fire  oontnd  radar,  the  AMRAAM  can  launch  up  to  ei^t  all-weather,  all-aspect 
radar-guided  missiles  beytmd  visual  range;  the  missiles  then  become  autonomous.  That  is, 
the  aircraft  radar  acquires  potential  targets,  tracks  them,  and  highlighta  them  for  the  pilot. 
If  the  pilot  decides  to  launch  a  missile  at  a  target,  the  system  feeds  initial  reference  data  on 
the  target  and  the  laundi  vdiide  into  a  computer  on  the  missile.  Following  laundi,  the 
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computer  guides  the  missile  to  a  midpoint  where  a  target  seeker  on  the  missile  becomes 
active  and  continues  to  guide  the  missile  to  the  target  Several  missiles  launched  by  the 
aircraft  can  do  this  simultaneously.  In  the  meantime,  the  pilot  is  firee  to  maneuver.  The 
criticality  of  the  integration  of  missile  and  aircraft  radar,  navigation,  control  and  display,  and 
other  systems  should  be  apparent* 

History 

The  AMRAAM  completed  its  initial  33-month  concept  phase  just  as  planning  for  MSIP 
began.  In  February  1979,  the  Air  Force  chose  Hu^es  and  Raytheon  from  five  competitors  to 
conduct  parallel  proof-of-concept  efforts,  leading  to  a  competition  between  them  for  the  full- 
scale-development  contract  In  December  1981,  Hu^es  won  the  contract  to  conduct  full- 
scale  development  throu^  1986.  Raytheon  later  reentered  the  program  as  a  second 
production  source  for  the  Hughes  design.  This  development  could  not  realize  the  bi^y 
optimistic  goals  set  for  it.  As  a  result,  the  AMRAAM  full-scale  development  was  unusually 
turbulent  A  aeries  of  schedule  revisicms  beginning  in  1984  moved  back  the  in-service  date 
for  production  missiles  firom  1986  to  1988.  In  1985,  the  development  program  conducted  an 
extensive  cost  reduction  ezerdse.*  In  its  efforts  to  integrate  AMRAAM,  MSIP  was  forced  to 
react  to  the  turbulence.  Difficulty  mating  the  missile’s  modular  rail  launcher  to  the  F-16 
further  complicated  MSIP. 

The  F-16  SPO  became  involved  in  the  AMRAAM  development  during  its  proof-of- 
concept  stage.  With  the  AMRAAM  JSPO,  the  F-16  SPO  developed  a  ’’minimum  proof  of 
concept”  definition  designed  to  ensure  that  no  high  technical  risks  related  to  the  int^ration 
of  aircraft  and  missile  would  remain  to  be  resolved  in  fuU-scale  development.  This  definition 
provided  a  basis  for  ground  simulation,  wind  tunnel  tests,  and  flight  tests.  Wind-tunnel  tests 
completed  in  late  1979  revealed  no  integration  problems.  Related  work  continued  into  the 
next  year,  and  led  to  the  need  for  General  Dynamics  to  develop  new  software  for  an  advanced 
control  interface  imit  (ACIU)  to  integrate  the  aircraft  and  missile.  Development  work  began 
on  that  software,  for  use  with  a  brassboard  ACIU  in  1981  proof-of-concept  tests  on  both 
AMRAAM  qrstems;  funding  limitations  ultimately  restricted  those  tests.  This  work  also  led 
to  extensive  exchange  of  technical  data  and  hardware,  under  contract,  between  General 
Dynamics  and  Westin^touse  (the  radar  contractor)  on  the  one  hand  and  the  missile 
contractors  on  the  other.  And  it  led  to  a  management  plan  for  integrating  the  F-16  and 
AMRAAM  during  the  AMRAAM  frill-scale  development 

»Prottir,  1986,  pp.  198-199. 

^Pretty,  1966,  p.  199.  For  more  detail  on  this  develoinnent  program,  see  Mayer,  finthccaning. 
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By  the  time  AMRAAM  fiill-acale  development  began  in  December  1981,  F-16  support 
for  the  program  was  firmly  integrated  with  MSIP  throng  a  contract-change  proposal,  CCP 
9140,  under  the  MSIP  contract.  Associate  contractor  arrangements  and  an  intmface-control 
working  group  were  quickly  established  among  Hughes.  Gmieral  Dynamics,  and 
Westin^iouse  as  part  of  the  new  developmmit  effort.  The  F-16  program  agreed  to  provide 
ei^t  test  F-16s  modified  to  support  AMRAAM  capability;  Westinghouse  agreed  to  modify 
APG-66  Block  IS  radars  to  support  ABCRAAM  capability  during  fuU-scale  development 
Development  of  the  ACIU  continued,  but,  by  June  1983,  the  AMRAAM  JSPO  concluded,  in 
concurrence  with  the  F-16  SPO  and  Genmal  Dynamics  that  it  would  not  be  available  in  time 
for  full-scale-development  tests.  The  JSPO  used  the  existing  central  interface  unit  instead. 

General  Dynamics  began  to  plan  for  production  incorporation  of  AMRAAM  in  early 
1983  and  for  its  integrated  system  performance  responsibility  for  AMRAAM  later  in  the  year. 
To  reduce  retrofit  costs.  General  Dynamics  planned  to  install  Group  A  wiring  provisions  on 
new  aircraft  beginning  at  Block  25A  (December  1984),  althouifo  full  AMRAAM  capability 
was  not  expected  to  be  available  until  Block  25D  (October  1985).  The  final  AMRAAM 
incorporation  was  then  expected  to  be  primarily  a  low-cost  software  insertion.  By  the  time  it 
was  approved,  the  European  Participating  Governments  had  expressed  an  interest  in  a 
similar  enhancement  for  their  aircraft.  The  F-16  SPO  arranged  for  a  limited  release  of  the 
common  flicht-control  computer  required  to  achieve  that  enhancement  for  the 
governments. 

By  early  1984,  it  was  dear  that  AMRAAM  would  not  be  available  when  expected, 
leading  MSIP  to  move  production  iiKorporation  back  to  Blo^  30B  (February  1987).  This 
block  became  part  of  a  larger  F-16C/D  reconfiguration  designed  to  better  coordinate 
hardware  and  software  production  incorporations.  The  AMRAAM  full-scale  development 
was  restructured  again  in  early  1985,  delaying  the  end  of  full-scale  development  until  April 
1988,  and  again  in  early  1986,  delaying  the  end  of  full-scale  development  to  late  1988.  MSIP 
held  to  its  production  incorporation  schedule  at  Block  30B,  incorporating  the  interfaces 
required  to  accept  the  missUe  whenever  it  became  available. 

With  this  incorporation  schedule  in  mind,  planning  for  the  Hu^ies  modular  rail 
launcher,  required  for  the  AMRAAM  system,  continued  during  this  period.  A  critical  design 
review  was  held  successfully  in  November  1984  and  a  production  contract  was  issued  in  June 
1986,  with  delivery  of 263  launchers  expected  during  March  to  October  1987.  That  contract 
proceeded  smoothly,  and  planning  was  initiated  for  a  follow-on  buy.  Then,  in  eariy  1987, 
developers  conduded  that  wei^t  growth  in  the  modular  rail  launcher  and  in  its  center  of 
gravity  was  not  compatible  with  the  F-16.  To  ensure  compatibility,  the  F-16  SPO  took 
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rMpondbility  for  the  necessary  redesign,  whidi  was  expected  to  delay  launcher  availability 
for  18  months.  Redesign  continued  in  late  1987.  By  then,  a  20-month  lead  time,  following 
establishment  of  a  stable  design,  would  be  required  to  start  production  and  an  additional  14 
months  worild  be  necessary  to  acquire  the  900  launchers  required  in  the  fleet.  By  early  1988, 
prototsrpes  were  oEpected  in  October  1988  and  initial  production  by  May  1989.  Difficulties 
continued  as  the  proposed  LAU-129  launcher  foiled  several  key  qualification  tests  late  in 
1988.  Problems  appeared  to  be  resolved  by  the  end  of  the  year,  with  initial  production  then 
expected  in  July  1989. 

Other  parts  of  the  development  went  more  smoothly.  The  first  full-scale-development 
miaailft  launch,  in  December  1984,  was  successful  in  the  sense  that  the  missile  passed  dose 
enouf^  to  the  target  to  destroy  it.  The  first  guided  launch,  in  May  1985,  was  similarly 
successful,  as  was  the  first  shot  firom  an  F-16  using  a  data  link,  in  September  1985.  This 
pattern  of  success  continued,  with  eadi  launch  becoming  more  difficult.  As  tests  on  the 
F-16/AMRAAM  integration  neared  completion  in  late  1987,  the  first  dual  launch  from  an 
F-16C  was  successful.  Despite  this  pattern  of  success,  problems  arose  in  late  1988  when  the 
system  was  first  tested  in  a  real,  multiple-target  environment. 

Two  more  technical  aspects  of  the  development  are  worth  noting.  One  concerns  radio- 
frequency  (RF)  compatibility.  As  noted  above,  tests  of  the  airborne  self-protection  jammer 
(ASPJ)  raised  the  possibility  of  serious  interference  with  radio  frequencies  important  to  the 
coordination  of  the  APG-68  and  AMRAAM  systems.  Efforts  to  achieve  RF  compatibility 
continued  into  1988. 

The  second  aspect  concerns  software  development  and  the  integration  of  software  with 
test  assets  as  development  improved  capabilities.  Because  software  was  advancing 
simultaneously  on  several  fixmts  relevant  to  AMRAAM,  achieving  compatible  software  for 
adequate  testing  was  a  continuing  problem.  Especially  relevant  systems  included  the 
APG-68  radar — ^whidi  itself  experienced  significant  instalnlity  and  growth  throu^  MSIP — 
the  central  interface  unit,  controls  and  displays,  and  the  ASPJ  and  advanced  interference 
blocker  unit  developed  to  reduce  interference  frtnn  ASPJ.  A  great  deal  of  imagination  was 
required  to  combine  various  generations  of  software  for  different  systems  and  infer 
performance  for  a  complete  set  of  software.  The  Systems  Integration  Laboratory  played  a 
central  role  in  this  part  of  the  AMRAAM  integration  ^ort. 

Disctission 

Ei^t  surprises  emmge  firom  the  above  discussion.  The  first  two  relate  to  the 
development  process  used  in  MSIP.  The  other  six  result  firom  technical  risk. 
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1.  Coordination  of  Sirftware  Development.  The  testiiig  of  B<^tware  and,  in 
particular,  the  integration  of  software  from  different,  subeystems  presented  a  challenge 
throu^out  MSIP.  MSIP  anticipated  the  challenge  by  maintaining  suf^rt  for  the  SIL  and 
the  F-16G/D  simulator.  It  also  used  a  system  of  software  Uocks  and  block  upgrades  that 
appeared  to  be  managed  in  a  coherent  and  fiurly  predictable  way.  But  tiie  official  history 
files  are  full  of  instances  when  the  software-integration  process  did  not  wcvk  as  wdl  as 
expected,  and  delays  and  conflicts  occurred  because  mature  software  firom  one  system  was 
not  available  to  test  another,  generally  causing  MSIP  managers  to  intervene  to  improvise  a 
creative  solution  to  a  software-development  problem.  Such  problems  may  be  especially 
prominent  for  AMRAAM  because  its  software  interfaces  with  other  systems  are  so  important 
to  MSIP  and  because,  with  delayed  delivery  of  the  missiles  themselves,  AMRAAM  software 
integration  may  have  received  more  attention  than  expected  early  in  MSIP. 

A  related  issue  is  the  willingness  to  retrofit  upgraded  software  to  replace  earlier 
versions  inserted  in  the  F-16C/D  as  placeholders.  The  direct  cost  of  retrofitting  software  is 
apparently  low  enou^  to  oicoorage  such  a  solution  when  software  matures  more  slowly 
than  expected.  However,  measured  in  terms  of  degraded  aircraft  capability  and  the  effects  of 
development  problems  during  the  period  when  upgraded  software  is  not  available,  the  cost  of 
this  solution  is  much  hi^er. 

2.  LbnitedFiuidingforEariyACIU  Testing.  MSIP  managers  proceeded  with  the 
ACIU  development  and  continued  to  rely  on  it  to  support  integration  efforts,  even  though 
funding  was  not  available  to  support  the  early  tests  planned  for  it.  Much  more  careful 
analysis  would  be  required  to  determine  whether  such  tests  might  have  identified  any  of  the 
problems  experienced  later. 

3.  Slow  Development  of  the  AC!IU.  MSIFs  quick  and  effective  substitution  of  the 
central  interface  unit  for  the  ACIU  is  a  good  illustration  of  a  principle  applied  many  times  in 
MSIP:  If  a  eystem  fails  to  work  as  expected  during  a  development,  an  effective  substitute 
should  be  available  to  replace  it  The  substitute  should  not  unduly  degrade  perfinmance  or 
complicate  testing.  Unlike  immature  software  blocks  substituted  in  a  similar  way  for 
unavailable  final  software,  this  substitution  does  not  appear  to  have  caused  serious  problems 
inBiSIP. 

4.  Late  Delivery  of  AMRAAM.  The  first  delay  in  the  AMRAAM  came  at  a  time 
when  infinmation  about  MSIP  as  a  whole  was  accumulating  rapidly.  That  information 
suggested  that  MSIP  had  been  somewhat  overoptimistic  about  how  quiddy  it  could  integrate 
subqrstems  into  a  new  F-16C/D.  Given  this  general  pattern,  MSIP  responded  reasonably  by 
readjusting  its  schedule  in  response  to  the  first  AMRAAM  delay. 
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6.  Still  Later  Delivery  of  AMRAAM.  Once  MSIP  decided  to  introduce  Group  A 
provisions  for  AMRAAM  early  and  retrofit  final  provisions  when  AMRAAM  was  available  for 
incorporation,  the  actual  availability  of  AMRAAM  assets  did  not  impose  mudi  constraint  on 
MSn*.  The  int^^tion  of  other  systems  did  not  appear  to  suffer  much  from  nonavailability 
of  AMRAAM  assets.  Therefore,  MSIP  had  less  need  to  resptmd  to  much  of  the  continuing 
turbulence  experienced  within  the  AMRAAM  program. 

6.  Pooriy  Mated  Modular  Rail  Launcher.  Given  the  apparently  smooth 
developmmit  of  the  AMRAAM  launcher,  the  late  discoveiy  of  inadequacies  in  it  is  a  puzzle. 
Presumably  the  developers  responsible  for  this  item  experienced  genuine  technical  surprise. 
Continuing  design  instability  is  even  more  puzzling.  The  F-16  SPO’s  decision  to  take 
responsibility  for  the  continuing  design  effort  presumably  promoted  integration.  Perhaps 
this  redesign  effort  did  not  receive  hi^er  priority  because  later  delivery  of  the  AMRAAM 
missiles  meant  that  a  delay  in  incorporating  launchers  would  have  little  effect  on  operational 
capability. 

7.  Operational  Problems  in  a  Compla  Target  Environment.  The  official 
historical  files  register  one  successful  test  firing  after  another  throuf^  MSIP.  They  provide 
no  basis  for  anticipating  any  trouble  latmr  in  the  program.  Some  observers  have  suggested 
that  early  tests  were  so  carefiilly  controlled  that  they  did  not  provide  realistic  data  on  the 
likely  performanoe  of  AMRAAM  when  it  was  fielded.  This  point  deserves  more  detailed 
examination. 

8.  RF  IncompatibiUty  with  ASP J.  AMRAAM  experienced  problems  with  ASPJ 
similar  to  those  discussed  in  the  history  of  the  AN/APG-68  fire-control  radar.  The  emphasis 
for  AMRAAM  is  more  on  incompatibility  and  less  on  a  shortage  of  ASPJ  assets  available  for 
testing,  but  all  issues  are  the  same. 

LOW-ALTITUDE  NAVIGATION  TARGETING  INFRARED  FOR  NIGHT  (LANTIRN) 

Description 

This  s]rstan  consists  of  two  pods  attached  to  the  exterior  of  the  F-16  and  a  head-up 
disphty  (HUD);  we  cover  the  HUD  below  and  focus  on  the  pods  here.  The  navigation  pod  uses 
a  wide-field-of-view  forward-looking  infiared  (FUR)  sensor  to  create  a  "night  window”  on  the 
terrain  bdow  that  the  pilot  can  view  throu^  the  HUD.  It  also  includes  terrain-following 
radar  that  supports  low-level  flight.  The  target  pod  includes  a  stabilized  wide-  and  narrow- 
fidd-of-view  targeting  FUR  that,  again,  provide  a  display  that  the  pilot  can  view  on  his 
HUD,  a  laaer/designator  finr  "painting”  targets  so  that  laser-guided  munitions  can  attack 
them,  and  an  electronic  hand-offunitfii»' coordination  with  Maverick  missiles.  Thepodscan 
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be  used  t<^ther  or  separately.  Used  tt^ther,  they  enable  an  attack  aircraft  to  use  a  safe, 
Burvivable,  low-level  penetration  route  to  approach  and  exit  a  target  area  and  acquire,  track, 
and  destroy  ground  targets  in  that  area  with  guided  or  nonguided  munitions,  day  or  night. 
The  prime  contractor  is  Martin  Marietta.^ 

History 

The  Air  Force  began  studying  tactical  applications  of  FLIR-based  systems  like 
LANTIRN  in  restricted  'lilack’’  pit^p-ams  in  the  1970s.  Those  efforts  yielded  a  formal  ‘Srhite” 
LANTIRN  program  in  December  1979,  just  as  the  F-16  MSIP  was  beginning.  The  new 
LANTIRN  program,  overseen  by  the  LANTIRN  SPO,  was  designed  to  support  sin^e-seat 
combat  aircraft,  the  A-10  and  the  F-16.  LANTIRN  was  expected  to  increase  a  sin^e  pilot’s 
ability  to  perform  complex  flying,  targeting,  and  weapon-delivery  tasks  simultaneously  in  a 
hostile  environment.  Early  on,  the  F-16  was  chosen  as  the  test  platform  for  the  LANTIRN, 
allowing  int^ration  efforts  to  b^in  early  as  the  pods  were  fitted  to  test  F-16  aircraft  and 
integrated  with  pre-MSIP  generations  of  ac.onics  and  software  in  that  aircraft.  Technical 
difficulties  in  the  LANTIRN  program  led  to  substantial  reductions  in  the  capability  of  the 
targeting  pod  and  to  delays  in  the  delivery  of  both  pods  to  MSIP.  First  production  of  the 
navigation  and  targeting  pods  finally  occurred  in  March  1987  and  July  1988,  respectively.^ 

The  initial  Air  Force  view  of  a  FLIR-based  system  on  an  F-16,  from  December  1979, 
placed  a  sin^e  600-pound  pod  on  the  right  side  at  the  F-16  engine  inlet  and  interfaced  this 
unit  with  a  wide-an^e  raster  HUD  and  an  HR  Maverick  air-to-ground  missile  system.  The 
pod  would  provide  automatic  target  recognition,  correlation,  and  launch  capability.  Both 
General  Dynamics  and  the  F-16  SPO  played  impOTtant  roles  in  defining  this  concept  and,  in 
particular,  in  defining  how  the  system  wotild  integrate  with  an  F-16.  The  request  for 
proposal  for  LANTIRN  full-scale  development  grew  out  of  this  work.  Martin  Marietta 
ultimately  won  the  full-scale-development  contract  in  September  1980. 

MSIP’s  serious  interest  in  integrating  LANTIRN  was  expressed  in  the  first  official 
contract  change  proposal  of  MSIP,  CCP  9101,  in  February  1981,  which  put  General 
Dynamics  on  contract  to  integrate  LANTIRN.  General  Dynamics  became  chairman  of  the 
LANTIRN  interfoce-control  working  group,  whidi  also  induded  Martin  Marietta,  Huid^es, 
and  Fairchild  Republic  and  met  regularly  with  the  LANTIRN,  A-10,  and  F-16  SPOs. 
Interfhce-control  documents  fin-  the  F-16/LANTIRN  pods  and  HUD  were  completed  by  July 

*Blak»,  1987,  p.  974;  General  Dynamics,  1990,  pp.  53-68. 

*For  mm  details  on  this  program,  see  Bodilly,  1993b. 


-82- 


1981.  In  late  1963,  General  Dynamiea  aisumed  integrated-system  performance 
responsibility  for  LANTIRN. 

During  tbe  LANTIBN  full-scale  development,  MSIP  provided  a  dedicated  F-16A  for 
testing.  FTii^t  tests  also  provided  data  for  the  F-16  fl4fot  manual.  General  D3mamic8 
provided  access,  under  contract,  to  its  Systems  Integration  Laboratory  (SIL),  where 
LANTIRN  pods  were  tested  and  prepared  for  fli^t  test.  Conflicts  among  the  LANTIRN, 
APG-68,  and  General  Dynamics  core  avionics  over  access  to  the  SIL  during  this  period  were 
resolved  in  favor  of  the  LANTIRN  program,  to  keep  it  on  schedule  for  its  flight  tests. 
Immature  aircraft  software  hampered  these  activities. 

LANTIRN  would  play  an  intend  role  in  giving  the  F-16  terrain-avoidance  and 
terrain-following  capabilities.  Hanning  for  an  advanced  terrain-avoidance  (ATA)  system  in 
May  1983  led  to  full-scale  development  that  was  dosely  coordinated  with  LANTIRN 
development.  Similar  coordination  with  the  development  of  the  digital  flight  control  system 
(DFLCS)  and  combined  altitude  radar  altimeter  (CARA)  was  similarly  important  to  this 
effort  The  F-16  SPO  reorganization  in  1983,  which  divided  MSIP  activities  between  two 
SPO  directorates,  YPD  and  YPR,  placed  these  activities  in  different  offices  until  ATA  and 
DFLCS  were  considered  mature  enough,  in  April  1984,  to  join  LANTIRN  activities  in  YPD. 
Efforts  to  integrate  these  capabilities  continued  through  1988.  Difficulties  in  any  one  of 
these  subsystems  could  slow  the  integration  of  all.  MSIP  developers  repeatedly  developed 
work-arounds  to  keep  integration  efforts  on  schedule.  General  Dynamics’  simulation  of  the 
F-16C/D  also  played  an  important  role  in  promoting  integration  among  these  ^sterns. 

The  close  relationship  of  the  LANTIRN  and  MSIP  developments  was  exemplified  hy 
the  dedaion  in  September  1985  to  combine  the  LANTIRN  and  F-16  combined  test  forces 
(CTFs)  at  Edwards  Air  Force  Base.  It  was  also  evident  in  early  1986,  less  positively,  when  a 
lack  of  ASPJ  hardware  in  F-16  test  aircraft  and  poor  representation  of  it  in  the  SIL  slowed  F- 
16  integration  testing  relevant  to  LANTIRN. 

More  generally,  the  intention  of  ASPJ  and  LANTIRN  caused  continuing  difficulties; 
spurious  radiation  firom  ASPJ  interfered  with  LANTIRN.  As  noted  above,  in  the  history  of 
the  AN/APG-68  fire-control  radar,  MSIP  began  development  of  an  advanced  interference 
blanking  unit  (AIBU)  to  overcome  this  problem.  Developers  also  separated  the  LANTIRN 
and  ASPJ  antennas  on  the  F-16  to  reduce  the  problem.  In  the  meantime,  using  only  one  of 
the  LANTIRN  and  ASPJ  systems  at  a  time  in  operatioiu  provided  another  solution,  but  one 
that  degraded  performance  coiunderably.  By  late  1966,  the  AIBU  and  an  RF-switchable 
notdi  filter  appeared  capable  iff  overcoming  these  proUema  As  development  difficulties 
grew  for  the  ASPJ,  however,  MSIP's  problem  became  less  one  of  «»iimin«ting  rf 
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incomimtibility  between  ASPJ  and  syetema  like  LANTIRN  and  more  (me  of  identifying  an 
alternative  to  ASPJ  and  asking  how  cmnpatible  LANTIRN  would  be  with  it.  For  example,  by 
mid-1986,  the  LANTIRN  and  F-16  SPOs  had  begun  oompatibilify  testa  between  LANTIRN 
and  ALQ-131  electronic  countermeasure  pods,  which  the  F-16  program  was  considering  for 
use  tmtil  ASPJ  development  was  complete. 

Problems  within  the  LANTIRN  pod  developments  themselves  significantly  delayed 
their  delivery  to  General  Dynamics.  Because  LANTIRN  is  so  central  to  the  operation  of  the 
Block  40  aircraft,  other  subqrstems  to  be  integrated  with  that  aircraft  experienced  difiBculties 
in  testing  when  LANTIRN  deliveries  were  delayed.  MSIP  implemented  dose  management  of 
the  few  LANTIRN  assets  available  to  limit  the  native  effects  of  such  shortages  on 
continuing  development. 

By  the  end  of  1988,  when  MSIP  had  expected  the  LANTIRN  pods  to  be  ready  for 
production  incorporation,  dedicated  MSIP  aircraft  had  provided  extensive  flight  testing  of  the 
two  pods.  The  tests  verified  that  the  navigation  pod  had  achieved  complete  integration 
capability  with  Block  40  aircraft.  Fli^t  test  of  a  production  navigation  pod  was  expected  in 
1989.  Fli^t  tests  of  the  targeting  pod  continued,  with  tests  of  a  production  targeting  pod 
expected  in  1989.  Contracts  for  fiill-rate  production  of  the  navigation  and  targeting  pods 
were  signed  in  November  1988  and  January  1989. 

Dtocusahm 

The  history  above  raises  three  problems  associated  with  shortages  of  test  assets,  two 
proUems  associated  with  the  development  process,  and  three  associated  with  technical  risk. 
We  have  already  discussed  a  number  of  them  above. 

1.  Conflict  over  Access  to  SIL.  As  noted  above,  excess  demand  existed  for  the  SIL. 
MSIP  managers  set  priorities  for  different  users  and  provided  access  on  the  basis  of  those 
prioritiee.  The  importance  of  LANTIRN  to  Block  40  presumably  contributed  to  the  hi^ 
priorify  it  received  for  access  to  the  SIL. 

ft.  Nonnvnilability  of  ASPJ  Assets.  The  nonavailabilify  noted  above  affected 
integration  of  the  LANTIRN  system.  LANTIRN  integration  also  suffered  fiem  a  poor 
representation  of  ASPJ  in  the  SIL.  Neither  problem  could  have  been  anticipated  without 
anticipating  tlw  difflculties  in  the  ASPJ  development  itself.  MSIP  managers  developed 
wodk-arounds.  As  it  became  increasingfy  apparent  that  ASPJ  mig^t  not  become  available, 
MSIP  began  to  consider  LANTIRN's  RF  oompatilnlity  with  other  dectronic  warfare  ssrstems. 

8.  Nonavailability  of  LANTIRN  Test  Assets.  Late  production  of  LANTIRN  pods 
complicated  the  integration  of  other  assets.  MSIP  developers  managed  this  problem  by 
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imposing  doee  controls  on  access  to  the  few  LANTIKN  pods  available  and  developing  work¬ 
arounds  when  pods  were  not  available.  Because  such  nonavailability  not  been 
anticipated,  no  special  {wocesses  were  available  to  facilitate  those  management  efforts. 

4.  Interdependencies  of  Terrain-Related  Snbasrateiiis.  The  subsystems 
associated  vrith  terrain  following  and  terrain  avoidance,  indmling  LANTIKN,  required 
especially  dose  integration.  It  is  not  surprising  that  their  integratkm  efforts  had  to  be 
dosdy  coordinated.  MSIP  managers  repeatedly  improvised  solutions  to  keep  those  efforts 
syndmnuzed. 

5.  Immatnre  Test  Software.  Interim  blocks  of  software  had  effects  on  the 
integration  of  LANTIRN  similar  to  those  noted  above  for  other  systems.  MSIP  managers 
dealt  with  them  in  a  mmilar  manner. 

6.  RF  Incompatibility  with  ASPJ.  LANTIKN  suffered  from  problems  similar  to 
those  discussed  above;  MSIP  managers  dealt  with  them  in  a  similar  manner. 

7.  Late  Production  Delivety  of  LANTIRN  Pods.  LANTIRN  pods  were  not 
actually  available  for  production  incorporation  in  the  first  Block  40  aircraft.  MSIP  prepared 
for  this  lack  by  incorporating  all  provisions  required  to  accept  the  pods  at  Block  40.  As  pods 
became  available,  they  could  then  be  retrofitted.  This  approach  had  effects  wimilar  to  those 
of  other  retrofits  discussed  above. 

8.  Limited  Capability  of  Targeting  Pod.  The  LANTIRN  targeting  pod  never 
achieved  the  full  capability  anticipated  for  it.  Because  that  problem  was  faced  and  resolved 
within  the  LANTIKN  program,  we  have  not  discussed  it  here.  Nonetheless,  it  is  true  that 
MSIP  did  not  receive  all  the  benefits  that  it  expected  in  the  beginning  firom  incorporating  the 
targeting  pods  into  the  F-16CVD.  Whoi  the  LANTIKN  withdrew  finom  its  early  performance 
goals  and  accepted  a  more  realistic  design,  M^P  continued  to  support  its  incorporation.  The 
new  targeting  pod  would  still  add  value  to  the  F-16(VD  design.  Perhaps  predictably,  the 
LANTIKN  designers  left  room  fi>r  a  module  that  misbt  provide  the  unrealized  capabilities  at 
some  future  date.  Incorporation  of  such  an  attenuated  tystem,  with  room  to  allow  future 
enhancement  throu^  a  retrofit,  is  entirely  consistent  with  MSIPs  approach  to  the  F-16C/D 
development 

HEAD-UP  DISPLAY  (HUD) 

UMCnpDOfi 

A  head-up  display  had  become  a  standard  part  of  the  codcpit  envinmmmit  befrne  MSIP 
began.  MSIP  envisioned  an  improved  version  ofthe  HUD  used  in  the  F-16A/B.  AHUDis 
essentially  a  window  placed  in  firont  of  the  pilot  so  that  while  looking  throu|^  it  at  the  scene 
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oatside  the  aircraft,  the  pilot  can  alao  read  information  (m  the  status  of  the  aircraft  and 
potential  targets,  prcgected  on  the  window  so  that  he  or  she  need  not  take  his  or  her  eyes  off 
things  onteide  the  aircraft.  The  HUD  also  allows  the  pilot  to  view  a  FUR-generated  picture 
of  terrain  on  the  window  to  support  flying  at  ni^t.  The  prime  contractor  for  this  system, 
which  the  F>16  used  in  several  variatioas,  was  initially  Marconi  Avionics,  Ltd.,  a  British 
company.  It  is  now  known  as  General  Electric  Company  Avionics,  Ud.,  still  a  British 
company.'' 

History 

The  initial  version  of  MSIP  included  provisions  for  a  wide-an^e  raster  (WAR)  HUD  in 
its  Stage  I.  It  would  display  a  FLIR-generated  image  to  the  pilot  to  support  manual  low-level 
flight  at  night-  This  WAR  HUD  became  known  as  the  LANTIRN  HUD  as  the  Air  Force’s 
plans  for  a  FUR  system  coalesced  into  the  LANTIRN  program.  Because  the  designer  and 
manufacturer  of  this  HUD  was  a  foreign  firm,  one  of  the  first  tasks  of  the  LANTIRN  HUD 
interface-control  working  group,  which  included  Marconi  and  General  Ilynamics,  was  to 
arrange  for  export  licenses  so  that  the  associate  contractors  could  exchange  technical  data. 

By  early  1981,  pilot  evaluations  of  a  simulated  cockpit  including  the  LANTIRN  HUD 
raised  concerns  about  the  'tunnel  effect”  of  the  displays,  including  the  HUD,  and  about  the 
HUD’s  protrusion  into  the  pilot's  ejection  envelope.  Follow-up  tests  on  the  Flight  Dynamics 
Laboratory  Advanced  Fights^  Technology  Int^ration  (AFTI)  testbed  confirmed  those 
problems.  MSIP  maintained  its  support  for  this  HUD,  however,  and  continued  its  test 
program.  By  late  1982,  that  program  was  revealing  significant  optical  design  problems  with 
the  LANTIRN  HUD.  Both  Systems  Command  and  Tactical  Air  Command  and  pilots  rated 
the  HUD  as  unsuitable  for  production  incorporation;  it  was  not  even  as  good  as  the  HUD 
used  in  the  F-16A/B.  Because  the  problems  were  severe  enough  to  threaten  the  planned  July 
1984  imshiction  incorporation  date  for  the  HUD,  MSIP  developers  began  looking  for  an 
alternative.  Th^  chose  the  wide-an^e  conventional  (WAC)  HUD  used  in  the  AFTI  aircraft, 
a  HUD  with  a  wider  ang^e  view  than  the  HUD  used  in  the  F-16A/B,  but  a  narrower  an^e 
view  than  the  LANTIRN  HUD.  Ihey  continued  develiqnnent  of  the  LANTIRN  HUD, 
intending  to  replace  the  WAC  HUDs  when  the  LANTIRN  HUD  completed  devel(q>ment. 
MSIP  managers  expected  that  the  development  of  the  LANTIRN  HUD  would  be  complete  in 
time  to  matdi  the  production  incorporation  LANTIRN  pods,  then  expected  in  1987. 

Efforts  to  integrate  tiie  WAC  HUD,  whidi  became  known  as  the  C/D  HUD,  began  in 
early  1983.  This  late  start  put  the  C/D  HUD  at  a  disadvantage  relative  to  other  subi^stems 
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scheduled  fw  incorporation  in  MSIP  Stage  II.  But  the  integration  went  smootiily.  In  March 

1984,  the  C/D  HUD  perfwmed  weU  on  the  first  fli^t  of  a  test  F-16  incorporating  nuuOT  MSIP 
avionics.  In  late  1984,  however,  it  failed  the  fiiU-scale-development  reliability  qualification 
test.  To  overcome  this  problem,  General  Dynamics  and  Marconi  agreed  to  continue  toward 
the  production  reliability  qualification  test  rather  than  start  the  full-scale-development  test 
over. 

In  the  meantime,  redesign  and  development  of  the  LANTIKN  HUD  had  proceeded. 

The  F-16  and  LANTIRN  combined  test  forces  at  the  Air  Force  Flight  Test  Center  held  a  fly- 
off  in  August  and  September  1984  between  the  CVD  and  LANTISN  HUDs.  On  the  basis  of  16 
flights,  the  pilots  preferred  the  C/D  to  the  LANTIRN  HUD.  As  a  result,  MSIP  decided  to 
equip  only  LANTISN-equipped  F-16s  with  the  LANTIRN  HUDs;  all  other  F-lBs  would 
recdve  C/D  HUDs.  Specifications  that  promoted  commonality  between  the  two  HUDs  early 
in  their  development  simplified  the  problems  of  maintaining  both  in  the  inventory. 

Production  management  of  the  LANTIRN  HUD  was  transferred  fiom  the  LANTIRN  to  the 
F-16  SPO  in  June  1985  to  continue  promoting  such  commonality. 

Unfortunately,  the  production  reliability  qualification  test  for  the  C/D  HUD,  begun  in 
January  1985,  revealed  problems  similar  to  those  discovered  in  the  earlier  qualification  test. 
MSIP  managers  considered  incorporating  the  HUD  as-is  to  maintain  the  schedule  and 
retrofitting  corrections.  The  HUD  ultimately  completed  its  qualification  test  in  December 

1985.  But  proUems  remained,  prompting  the  managers  to  incorporate  an  incomplete  qrstem 
at  Block  SOB  (February  1987)  as  planned  and  retrofit  necessary  corrections  later.  In  January 
1988,  engineering-change  proposals  were  accepted  to  correct  the  reliability  problems  in  the 
CS/D  HUD  and  to  retrofit  an  important  mimring  capability. 

Development  of  the  LANTIRN  HUD  continued  with  few  difficulties.  In  late  1986, 
problems  developed  in  efforts  to  integrate  the  enhanced  envelope  gun  si^t  (E%S)  with  the 
LANTIRN  HUD.  Developers  prepared  to  drop  thin  integration  effort  if  necessary  to  maintain 
their  schedule.  MSIP  tested  E%IS  software  in  the  Systems  Integration  Laboratory  to  isolate 
the  source  of  these  difficulties.  A4justments  corrected  the  problem,  aDowing  fli^t  tests  of 
E^GS  at  Edwards.  Additional  problems  with  the  glare  shield  for  the  HUD  developed,  but  the 
develoinnent  as  a  whole  remained  on  schedule  for  incorporation  at  Block  40  in  December 
1988. 

All  these  effints  aimed  at  placing  a  sinf^e  HUD  in  each  aircraft  In  1986,  the  Tactical 
Air  Command  and  Air  Force  Systems  Command  headquarters  omcluded  that  an  additional 
HUD  should  be  added  to  F-16Ds  to  promote  LANTIRN  training,  'niis  aft-seat  HUD  monitor 
would  repeat  the  video  image  seen  on  the  primary  HUD  in  fiont  In  late  1987,  it  was  added 
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to  th«  principal  contracriial  vehicle  for  Block  40,  CCP  9226,  and  scheduled  fiw  production 
incorparation  at  Block  40C  and  retrofit  into  Blo^  40A  and  40B.  Its  development  proceeded 
without  difficulty.  A  similar  requirement  was  added  to  what  would  become  Block  50. 

Dtoeusaion 

The  history  above  identifies  four  surprises  associated  with  technical  risk  and  one 
associated  with  system  spedficatioiL 

1.  First  Rejection  of  the  LANTIRN  HUD.  Very  early  in  MSIP,  test  pilots  raised 
objections  to  the  LAMTHtN  HUD.  MSIP  took  them  under  fulvisement  but  continued 
development.  Later,  a  different  set  of  problems  arose.  MSIP  would  not  have  avoided  these 
later  problems  even  if  it  had  reacted  more  aggressively  to  the  earlier  concerns.  Its  reaction  to 
the  new  concerns  reflected  well  the  general  MSIP  approach  to  surprises.  Given  the  potential 
that  adHitinnal  develi^ment  work  on  the  HUD  would  threaten  the  MSIP  schedule,  MSIP 
managers  sou^t  an  interim  placeholder  for  the  HUD.  They  took  advantage  of  the  derivative 
nature  of  the  HUD  by  turning  to  a  dose  variation  produced  by  the  same  firm.  Having  found 
a  satisfactory  substitute,  they  then  set  up  an  effective  competition  between  the  two  HUDs. 
Because  a  single  company  designed  and  produced  both,  the  competition  was  limited  in 
economic  terms.  But  this  aggressive  approach  effectively  took  advantage  of  the  flexible  MSIP 
environment  to  reopen  the  specifications  for  the  F*16C/D.  MSIP  used  even  more  aggressive 
competitions  elsewhere  to  address  similar  problems. 

2.  A  Reliability  Test  Failure.  When  the  C/D  HUD  failed  its  reliability  test,  MSIP 
again  responded  flexibly.  It  resolved  to  eliminate  the  reliability  problem,  but  was  prepared 
to  introduce  a  less-than-final  HUD  to  maintain  its  schedule.  Such  an  introduction  was 
ultimately  required;  future  retrofits  would  implement  full  capability. 

3.  Second  Rejeetioa  of  the  LANTIRN  HUD.  The  cmnpetition  confirmed  that  the 
LANTIRN  HUD  had  not  met  the  specification  expected  early  in  the  program.  BASIP 
responded  with  a  creative  change  in  the  system  specification:  The  F-16C/D  program  would 
now  support  two  HUDs.  MSIP  undertook  measures  to  limit  the  cost  of  that  support,  and  it 
would  accept  HUDs  less  capable  than  those  initially  expected.  But  the  program  would 
maintain  its  basic  schedule  fhr  Block  40. 

4.  Difficulty  Integrating  the  When  difficulties  arose,  MSIP  undertook 

aggressive  devdopment  activity  to  resolve  them.  But  managers  were  fully  prepared  to  drop 
this  capability  if  it  threatened  their  basic  schedule. 

6.  Adding  an  Aft  Sent  HUD  Monitor.  As  with  other  changes  in  system 
specifications,  this  one  proceeded  smoothly.  It  came  late,  presumably  because  designers  and 
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UMTS  could  not  fitlly  q>preeiate  its  usofiilnoat  until  they  had  some  experience  with  the 
LANTIRN  system.  MSIP  was  well  straetured  to  allow  an  dfective  response  to  sudi  learning. 

GLOBAL  POSITIONING  SYSTEM  (GPS) 

Doaer^ition 

GPS  is  a  space-based  radio  positioning,  navigation,  and  time-transfer  system 
composed  of  three  basic  segments.  The  space  segment  includes  many  satdlites  that 
continuously  broadcast  information  that  can  be  used  to  provide  instantaneous  and  precise 
location.  The  control  segment  includes  numerous  monitor  stations  and  ground  antennas 
located  around  the  world  to  coordinate  the  satellites.  The  user  segment  includes  user 
equipment  sets  that  derive  navigation  and  time  information  from  satellite-transmitted  data 
for  use  on  a  host  vehicle.*  A  Joint  Program  Office  that  included  many  defense  and 
nondefense  agencies  wianngaJ  the  development;  the  Air  Force  has  played  the  role  of  first 
am<mg  many  equals  in  that  office.  The  F-16  was  only  one  of  many  planned  host  vehicles  in 
the  user  segment  Hie  F-16  program  coordinated  the  integratim  of  GPS  user  equipment  into 
the  F-16C/D.  MSIP  concerned  itself  primarily  with  the  user-segment  portion  of  the  GPS 
program.  Availability  of  satellites  during  MSIP,  however,  also  affected  MSIPs  ability  to 
integrate  GPS  fully  into  the  F-16C/D. 

By  improving  location  data  and  providing  it  in  real  time,  GPS  was  expected  to  support 
the  efforts  of  the  F-16C/D  associated  with  LANTIRN,  which  allowed  low-altitude  flying  and 
hi^-predsion  attack  of  ground  targets.  Testing  revealed  that  it  reduces  an  F-16  pilot’s 
workload  associated  with  night  terrain  following  by  50  to  75  percent  in  a  benign  environment 
and  by  90  percent  in  a  high-threat  environment* 

Hlstoiy 

Hie  relationship  between  the  GPS  and  F-16  programs  appeared  to  b^n  smoothly. 
OSD  approved  a  development  program  for  GPS  in  1973;  full-scale  development  began  in 
1979,  and  development  testing  of  user  equipment  began  in  1983.  Integration  with  the  F-16 
was  well  under  way  before  General  Dynasmics  and  the  F-16  SPO  even  began  to  talk  about 
MSIP.  Hie  F-16  SPO  used  contract-diange  proposals**  outside  the  normal  MSIP  series  to  set 
up  its  OPS  integration  work.  But  the  office  in  the  SPO  responsible  for  MSIP  at  that  time, 
YPR,  also  oversaw  this  contract  diange  proposal.  Hie  GPS  JPO  awarded  Magnavox  and 

■Navstar  OPS  JPO,  1982;  for  moro  information  on  the  space  segment,  see  Webb,  unpublished. 

•Clark,  luL 

lOCCP  5821  for  the  initial  phase  of  integration,  5430  for  F-16  installation  design,  5535  and  5536 
for  intagratkm  and  flight  test  on  the  F-16.  CCPs  added  later  also  lie  outside  the  nomal  MSIP  series. 
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Rockwell  C<dlixi8  fuU-scale-development  contracts  for  user  equipment  in  late  1979;  the  two 
contractors  coordinated  with  the  F-16  SPO  immediately,  and  TAG  quickly  provided  F-16 
aircraft  for  test  purpoees.  Development  testing  and  evaluation  would  end  in  April  1985  with 
a  demonstratimi  of  competing  user  systems,  which  Rockwell  won,  clearing  the  way  to 
int^rate  this  equipment  for  production  incorporation  in  Block  40  aircraft  starting  in 
December  1988. 

Integration  problems  began  in  1981.  GPS  funding  problems  led  to  delays  in 
integration  testing.  And  problems  with  the  Kalman  fitter  that  integrated  alternative  sources 
of  location  data  arose  and  persisted.  The  F-16  SPO  continued  work  on  integration,  using 
insights  firom  the  work  it  had  done  to  integrate  the  precision  location  strike  qrstem  (PLSS). 
In  the  meantime,  in  early  1982,  the  F-16  program  management  directive  dropped  GPS  firom 
the  MSIP  configuration  as  too  rist^  to  include  at  that  time.^^  The  Air  Force  reinstated  it 
later  in  the  year,  but  moved  its  production-incorporation  date  back  15  months  to  December 
1988.  This  change  disrupted  the  GPS  JPO’s  planning. 

Among  the  many  integration  issues  that  arose,  three  stand  out.  First,  in  early  1982, 
int^ration  agreements  were  revised  to  add  software  modifications  that  would  provide 
bombing  algorithms,  moving  way-point  steering,  self-contained  airborne  instrumented 
landing  approach,  and  GPS  backup  steering.  That  is,  the  location  and  time  data  provided  by 
GPS  could  support  each  of  these  functions.  Those  modifications  effectively  automated  such 
support.  Later  modifications  would  make  similar  changes,  effectively  making  applications  of 
data  firom  GPS  part  of  the  the  GPS  introduction  to  the  F-16.  Such  integration  appears  to 
have  proceeded  without  serious  difficulty. 

Second,  in  early  1984,  developers  found  that  insufficient  space  remained  in  the 
avionics  suite  to  accommodate  and  integrate  GPS,  PIiSS,  and  the  Enhanced  JTIDS  (EJS) 
communications  system.  The  Air  Force  set  up  an  independent  review  team  to  assess  and 
resolve  the  problem.  The  use  of  such  teams  was  unusual  during  MSIP,  suggesting  that  this 
issue  presented  especially  difficult  problems.  The  team  generated  two  alternatives  later  that 
year.  The  one  chosen  altered  the  PLSS  vehicle  navigation  sjrstem  (VNS)  to  create  an 
adaptive  targeting  data  link  (ATDL)  that  could  provide  data  transfer  capability  for  PLSS  or 
JSTARS  and  could  be  integrated  with  the  navigation  capabilities  of  GPS.  It  left  EJS  as  a 
stand-alone  system,  initiating  a  new  effiirt  to  develop  the  ATDL  in  the  PLSS  SPO,  under 
F-16  SPO  oversight,  and  resolving  the  integration  problem  within  the  F-16  itsdf. 

^^U.S.  Air  Force,  Headquarters,  1984,  Revision  18. 
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This  was  the  hi^iest  visibility  problem  to  arise  in  int^ating  GPS  with  other 
subsystems.  Other  subiqrstem  integrations  explicitly  highlighted  in  ofiSdal  historical 
documents  include  those  between  GPS  and  the  AMRAAM,  APG-68,  HAVE  QUICK  radio, 
inertial  navigation  system,  antennas,  and  systems  associated  with  terrain  following  and 
avoidance. 

Third,  in  early  1986,  the  F-16  SPO  discovered  that  the  GPS  JPO  had  not  been 
pursuing  design  and  production  of  mounting  racks  for  the  GPS  as  they  had  agreed.  The  F-16 
SPO  had  great  difficulty  getting  the  GPS  JPO  to  specify  mounting-rack  requirements  to 
Rockwell  Collins.  A  critical  design  review  was  finally  held  in  June  1987,  raising  a  few  final 
questions  about  this  problem  and  resolving  it. 

GPS  was  ultimately  successfully  integrated,  incorporated  into  Block  40,  and  will  be 
retrofitted  into  Blocks  25, 30,  and  50  aircraft  But  integration  testing  continued  beyond  the 
December  1988  deadline  for  Block  40.  The  tests  took  longer  than  expected  because  the  full 
constellation  of  satellites  for  the  GPS  had  not  been  launched.  Because  the  location  accura(7 
that  can  he  achieved  with  GPS  depends  on  how  many  satellites  are  within  line  of  si^t  of  the 
user  equipment,  enough  satellites  were  rarely  available  at  the  test  site  to  test  GPS  fully. 

Test  schedules  had  to  be  carefully  arranged  to  take  advantage  of  the  limited  constellation  of 
satellites  available  for  testing. 

DiscuMion 

Six  surprises  come  up  in  the  course  of  this  history.  The  first  is  more  a  decision 
milestone  than  a  problem.  The  next  three  conrom  problems  of  coordinating  the  F-16  and 
GPS  programs.  The  last  two  surprises  concern  technical  risk. 

1.  Rockwell  Collins  Wins.  As  the  date  for  choosing  between  Rockwell  Collins  and 
Magnavox  approached,  the  F-16  program  negotiated  complete  contract-change  proposals  on 
the  MSIP  contract  tailored  to  each  of  the  contractors.  Hence,  as  soon  as  the  choice  was  made, 
the  F-16  program  was  prepared  to  proceed  inunediately. 

2.  Early  instability  in  GPS  Program.  As  uncertainty  arose  in  the  F-16  program 
about  the  stability  of  GPS,  it  was  predictable  that  the  F-16  program  would  back  away.  To 
limit  tbe  already  substantial  risk  associated  with  multiple,  parallel  integrations,  the  MSIP 
sou^t  to  include  lower  risk  subsystem  developments  in  the  F-16C/D  configuration.  As 
constraints  on  space,  wei^t,  or  other  dimensions  forced  MSIP  to  choose  among  subsystems 
to  include  in  a  particular  configuration,  the  relative  levels  of  risk  associated  with  individual 
systems  figured  prominently  in  the  deliberations.  As  MSIP  apinoadied  production 
incorporation  at  Stage  n,  the  SPO  even  divided  its  activities  into  those  mature  enough  to  be 
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managed  as  production-oriented  programs  and  those  that  should  still  be  considered 
development  programs.  Hence,  the  F-16  program’s  reaction  to  instabiliiy  in  the  GPS 
program  is  consistent  with  its  broader  attitude  inward  risk.  What  exactly  tipped  the  balance 
to  first  exclude  the  GPS  program  and  then  fairly  quickly  readmit  it  is  less  clear.  Perhaps 
because  MSIP  did  not  have  to  find  a  placeholder  for  GPS  if  it  was  not  included,  MSIP  could 
handle  GPS  more  flexibly  than  other  risl^  pn^rams. 

3.  Problems  with  Mounting  Racks.  Technologically,  mounting  racks  may  be  the 
least  challenging  part  of  the  GPS  system;  therefore,  it  might  not  be  surprising  if  they  were 
overlooked.  Without  them,  however,  the  F-16  could  not  incorporate  the  GPS  system — a  fact 
that  was  probably  clearer  to  the  F-16  program  than  to  the  GPS  program.  Although  proper 
integration  of  the  racks  and  GPS  hardware  would  most  likely  succeed  if  the  GPS  program 
took  responsibility  for  these  racks,  the  F-16  program  may  have  been  better  suited  to  address 
what  was  egaentially  a  system-integration  issue.  In  any  case,  the  problem  was  not  resolved 
until  the  F-16  SPO  acted.  It  apparently  had  to  act  with  some  conviction  to  resolve  a  problem 
that  its  managers  believed  had  already  been  adequately  planned  for  in  an  earlier  agreement. 

4.  A  Shortage  of  Satellites.  The  number  of  satellites  to  be  included  in  the  final 
constellation  of  the  GPS  program  and  the  schedule  for  launching  them  varied  over  the 
program  life.  This  instability  resulted  mainly  firom  factors  beyond  the  control  of  the  GPS  and 
F-16  programs.  Therefore,  these  programs  could  not  avoid  uncertainty  about  this  factor. 
Given  the  uncertainty,  all  the  F-16  managers  could  do  is  what  they  did.  This  kind  of  problem 
again  emphasizes  the  importance  of  good  managers  in  an  uncertain  development 
environment. 

6.  Problems  with  the  Kalman  Filter.  Such  technical  problems  could  easily  be 
expected  in  an  integration.  They  persisted  for  some  time  but  do  not  appear  to  have  presented 
any  serious  problems.  The  opportunity  to  draw  lessons  fiom  a  program,  PLSS,  that  used  a 
Kalman  filter  in  a  similar  way  illustrates  a  (probably  unexpected)  advantage  of  dealing  with 
many  parallel  integrations. 

6.  Conflict  in  the  Avionics  Suite.  Although  we  should  not  have  expected  the  MSIP 
to  predict  this  precise  conflict,  it  is  the  type  of  problem  one  would  expect  in  an  integration 
program.  Independent  review  teams  were  a  standard  part  of  the  F-16  SPO  repertoire  for 
managing  such  problems.  The  team  used  in  this  conflict  appears  to  have  resolved  the 
problem  quickly  and  effectively.  Standard  procedures  handled  this  surprise  without 
difficulty.  Also  worth  noting  is  MSIP’s  willingness  to  initiate  a  new  development  program  as 
part  of  the  solution  to  the  problem,  a  program  that  would,  if  successfiil,  change  the 
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spedfications  for  the  F-16C/D  under  development.  Such  flezilHlity  allowed  an  independent 
team  to  consider  a  broad  range  of  potential  solutitms. 

ALTERNATE  RGHTER  ENGINES:  F100-PW-220  ANDF11(MSE-100 
DMcription 

Alternate  Fighter  Engines  (AFEs)  are  hi^-performance  fighter  engines  that  became 
interchangeable  in  the  F-16C/D,  which  is  not  to  say  that  they  have  equivalent  performance. 
The  FllO-GE-100,  designed  and  built  by  General  Electric,  has  sli^tly  hi^er  thrust  than 
Pratt  and  Whitney’s  FlOO-PW-220,  especially  when  paired  with  a  modular  common  inlet  duct 
(MCID)  and  large  forward  input  module  (FIM),  both  developed  as  part  of  MSIP.  These 
modifications  increase  the  airflow  available  to  the  FllO-GE-100,  thereby  increasing  the 
thrust  that  it  can  generate.  But  relative  to  the  engine  used  in  the  F-16A/B,  the  FIOO-PW- 
200,  the  AFEs  display  significantly  improved  opm-ability,  safety,  and  supportability; 
significantly  longer  lives;  and  less  performance  loss  over  those  lifetimes.  And  the  two  are 
interchangeable  in  the  configured  engine  bay  (CEB)  developed  during  MSIP  to  accommodate 
them  in  the  sin^e-engine  F-16C/D.^ 

The  Propulsion  SPO  oversaw  the  development  of  these  government-furnished  systems 
and  continued  to  supervise  their  production.  It  also  oversaw  the  annual  production 
competition  that  determined  the  splitting  of  the  Air  Force’s  annual  buy  of  engines  for  the 
F-15  and  F-16  between  the  two  engines.  The  first  annual  competition  was  held  in  1984, 
generating  the  result  that  all  engines  purchased  that  year  for  the  F-16  would  come  fiom 
G  ■’  t  era!  Electric.  Pratt  and  Whitney  was  more  successful  in  later  competitions,  leading  the 
F-16  SPO  to  int^rate  both  designs  in  the  F-16C/D  aircraft  that  it  was  developing  in  MSIP. 

History 

Development  efforts  for  these  new  engines  b^an  at  just  about  the  same  time  that 
General  Dynamics  and  the  F-16  SPO  began  to  discuss  MSIP.  Both  engines  began  as  efforts 
to  develop  new  derivative  engines  based  on  other  engines  already  in  the  inventoiy.  Although 
General  Electric  had  been  working  on  the  engine  that  evolved  into  the  FlOO-GE-lOO  for  some 
time,  the  Air  Force  only  began  to  fund  that  effort  in  February  1979.  The  effort  attempted  to 
use  the  FlOl  engine  developed  by  General  Electric  for  the  B-1  as  the  basis  for  deriving  a 
longer  lived,  more  operable  filter  engine  that  could  compete  with  or  conceivably  replace  the 
F1(X).  In  July  1979,  the  Air  Force  began  to  fund  the  development  work  that  led  to  the 

i^For  additional  details  on  the  development  of  these  subeystema,  see  Camm,  1993. 
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FlOO-PW-220,  w<H^  that  sou^t  modifications  to  the  FIDO  engines  used  in  the  F-15  and  F-16 
to  improve  their  operability  and  lifetimes. 

By  tile  end  of  1980,  the  General  Electric  prototype  had  already  executed  extensive  test 
flights  on  an  F-16,  with  positive  results.  Early  test  flints  for  modified  versions  of  the  Pratt 
and  Whitiity  FlOO  were  more  likely  to  take  place  on  the  two-engine  F-15.  Its  two-engine 
design  facilitated  safe  engine  testing,  because  it  could  always  use  a  standard  engine  in 
tandem  with  an  engine  to  be  tested.  Onty  because  the  F-15  could  not  accommodate  early 
versions  of  the  General  EUectric  engine  was  it  tested  on  an  F-16  as  early  as  it  was. 

The  F-16  SPO’s  serious  involvement  with  these  engines  began  as  it  became  involved  in 
preparations  for  the  Alternate  Filter  Engine  competition  in  1984.  Its  directorate  of 
development  programs,  YPR,  played  an  active  role  in  planning  the  integration  of  both 
engines  in  the  F-16  airfiame  at  that  time.  Hie  idea  of  designing  a  common  engine  bay  to 
enable  any  F-16C/D  to  use  either  engine  grew  out  of  this  integration  effort.  Design  work  that 
would  lead  to  the  configured  engine  bay  (CEB)  began  in  YPR  in  late  1983.  The  CEB  was 
approved  for  incorporation  of  the  FllO-GE-lOO,  selected  for  the  F-16  in  the  February  1984 
competition,  shortly  after  the  selection.  In  June,  that  engine  flew  in  an  MSIP  aircraft  for 
the  first  time. 

By  the  end  of  1984,  the  F-16  SPO’s  test  F-16XL-2  aircraft  was  testing  the  FllO-GE- 
100  with  an  enlarged  inlet  to  take  advantage  of  its  fiiD  thrust.  Planning  began  for  a  modular 
inlet  that  would  allow  such  a  capability  on  the  F-16C/D.  By  early  1985,  the  CEB  had  been 
scheduled  for  production  incorporation  at  Block  30  (June  1986);  an  enlarged  inlet  had  been 
scheduled  for  production  incorporation  in  Block  30B  (February  1987).  By  the  end  of  the  year, 
inlet  design  had  evolved  to  a  modular  common  inlet  duct  (MCID)  that  could  optimize  air  flow 
for  the  engine  installed  at  production.  It  was  to  be  incorporated  at  Block  SOD  (October  1987) 
with  a  large  forward  inlet  module  (FIM)  for  the  FlOO-GE-100.  Work  on  a  small  FIM  for  the 
FlOO-FW-220  continued. 

In  the  meantime,  a  preproduction  FlOO-PW-220  flew  in  an  F-16  for  the  first  time  in 
May  1985.  These  Pratt  and  Whitney  engines  would  be  incorporated  for  the  first  time  at 
Block  30C  (June  1967).  The  portion  of  the  block  receiving  this  engine  then  became  32C  to 
distinguish  these  aircraft  firom  those  with  General  Electric  engines. 

By  1986,  production  of  Alternate  Flatter  Engines  was  well  under  way.  Production 
proceeded  more  slowly  than  expected  and  many  quality  and  manufiscturing  problems  were 
uncovered,  attracting  hi^-level  oonoem  at  the  F-16  SPO  and  Aeronautical  Bystems  Division. 

**Pratt  and  Whitney  also  woo  a  share  of  the  competition,  but  all  its  engines  would  be  used  in 
P-15s.  Later  buys  rfIVatt  and  Whitney  engines  would  be  used  in  the  F-16,  as  well. 
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The  F-16  SPO  worked  daily  with  the  Propulsion  SPO  to  overcome  such  problems.  General 
Dynamics  was  forced  to  use  extensive  overtime  to  make  the  first  FlKHlE-lOOs  delivered 
acceptable  for  Block  30  aircraft.  By  the  end  of  1986,  however,  both  engine  suppliers  had 
achieved  their  contractually  agreed*upon  delivery  schedules.  Although  problems  severely 
complicated  production  of  new  F-lGs,  their  ^ects  on  MSIP  were  limited.  Late  engines  do  not 
appear  to  have  hampered  other  development  ^orts  under  way  in  MSIP. 

One  problem  that  emerged  at  this  stage  can  be  linked  directly  to  an  integration 
problem.  In  late  1985,  test  pilots  began  to  notice  a  ‘1;hump”  during  fli^t  ,  which  they 
associated  with  the  FllO^E-100  engine.  Extensive  tests,  conducted  jointly  by  the  F-16  and 
Propulsion  SPOs  and  their  contractors,  could  not  identify  the  problem;  in  the  meantime,  the 
flight  of  F-16s  was  carefully  circumscribed  to  avoid  thump.  A  process  of  elimination  finally 
determined  that  the  problem  did  not  come  firom  the  engine  per  se  but  firom  the  way  in  which 
the  engine,  inlet,  and  airframe  were  integrated.  Under  certain  flight  conditions,  a  shock 
wave  formed  in  the  inlet  and  manifested  itself  by  creating  a  distinct  thump.  When  this 
phenomenon  was  finally  understood,  developers  concluded  that,  although  disconcerting,  it 
was  not  dangerous.  They  would  continue  to  search  for  solutions,  but  the  search  no  longer 
had  the  urgency  assumed  when  pilots  first  reported  thumps.  The  FlOO-PW-220  experienced 
no  similar  integration-related  problems. 

Devdopment  woik  continued  on  the  MCID.  Flight  testing,  which  began  in  late  1987, 
discovered  severe  vibrations  during  hi^-speed  maneuvers,  particularly  in  a  hi^  an^e  of 
attack.  Fli^t  tests  continued  into  1988  without  resolving  the  problem.  During  that  period, 
most  devdopment  interest  shifted  to  the  next  generation  of  derivative  engines,  the  improved 
perfinrmance  engines,  that  would  be  incorporated  at  Block  50.  The  F-16  SPO  was  actively 
involved  in  planning  for  the  design,  integration,  and  production  incorporation  of  these 
engines  fimn  the  beginning  of  their  development  That  is,  it  became  involved  much  earlier 
than  it  had  with  the  Alternate  Filter  Engines. 

DiacuMion 

Thia  histoiy  reveals  five  surprises  associated  with  the  integration  of  the  Alternate 
Fifiditer  Engines.  These  surprises  take  on  a  smnewhat  different  cast  fit>m  those  for  othm- 
qrstems,  perhaps  because  the  enginedevdopment  programs  entered  MSIP  later  than  those 
for  the  five  subsystems  described  above  and  presented  a  different  kind  of  integration 
inoUem.  The  first  three  conoem  new  information  on  what  subsystems  would  be  integrated. 
The  last  two  stem  firom  technical  risks. 
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1.  Hie  Alternate  Fii^ter  Engine  Competitioii.  The  announoement  of  this 
competition  presented  MSIP  with  the  prospect  of  having  to  accept  either  or  both  of  two  new 
engines  that  were  designed  with  only  limited  MSIP  input.  MSIP  participated  actively  in  the 
preparations  for  the  cmnpetition,  but  by  then,  the  designs  for  both  engines  were  fairly  well 
defined.  The  nature  of  the  competition  presented  MSIP  with  the  prospect  of  continuing 
uncertainty;  Each  year,  it  mi^t  be  required  to  integrate  a  difierent  kind  of  engine  chosen  by 
someone  else.  For  the  competition  to  be  effective,  the  F-16  SPO  could  exercise  only  limited 
control  over  source  selection.  MSIFs  response  to  this  persistent  uncertainty  was  to  develop 
the  CEB,  a  device  that  would  minimize  the  effects  of  uncertainty.  It  represented  the  first  of  a 
series  of  large  steps  that  MSIP  would  take  to  int^rate  these  engines  as  it  learned  more 
about  them.  The  standard  processes  within  MSIP  readily  provided  the  flexibility  required  to 
initiate  and  execute  the  CEB  development  effort. 

2.  Selection  of  General  Electric.  Once  General  Electric  was  chosen  as  the  sole 
source  of  new  engines  in  the  first  year  of  the  competition,  MSIP  could  begin  to  focus  on 
engine  integration.  MSIP  sought  to  exploit  the  principal  advantage  of  the  General  Electric 
engine  over  its  rival,  its  thrust,  by  initiating  developments  to  increase  the  inlet  size  for  those 
F-IGC/Ds  that  would  use  the  General  Electric  extgine.  To  maintain  commonality  between  the 
engines  that  mi^t  be  used  in  the  future,  it  lued  an  approach  that  focused  on  a  module  to 
accommodate  the  General  Electric  engine  first.  Again,  MSIP  was  well  structured  to  admit 
the  specification  changes  and  execute  the  development  efforts  required  to  realize  them.  It  is 
worth  noting  that,  as  development  and  learning  about  the  integration  issues  associated  with 
the  new  engine  proceeded,  MSIP  was  able  to  a<yu8t  its  design  and  schedule  without  much 
difficulty. 

8.  Sdection  of  Dual  Sources.  Perhaps  inevitably,  the  time  came  when  MSIP  had  to 
accept  two  separate  engine  designs.  The  CEB  changes  anticipated  that  date.  When  it 
became  real,  MSIP  could  begin  to  focus  on  introducing  an  inlet  for  the  Pratt  and  Whitney  as 
welL  It  also  modified  its  block  structure  to  identify  separate  miniblocks  with  General 
Electric  and  Pratt  and  Whitney  engines.  Although  this  last  change  was  made  primarily  at 
the  request  of  logistics  planners  concerned  about  the  growing  variefy  of  F-lfifVDs  that  they 
would  have  to  support,  it  also  helped  MSIP  manage  its  affedrs  in  the  presence  of  separate 
engine  designs. 

4.  Hminp.  When  test  pilots  first  nolaoed  thump,  they  did  not  associate  it  with  the 
airframe.  As  a  result,  earfy  attempts  to  deal  with  it  occurred  in  the  engine  program.  Only 
when  efforts  thme  frdled  to  resolve  it  did  the  possibility  of  an  integration  problem  become 
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apparent  Even  tiien,  MSIPs  role  in  resolving  the  problem  was  limited.  The  Propulsion  SPO 
ultimateiy  readied  the  determination  that  it  was  not  dangerous. 

6.  Vibration  in  the  MCID.  MSIP  was  much  more  engaged  in  resolving  this  problem, 
which  was  detected  in  early  fli^t  tests  of  an  MSIP-developed  subsystem.  Our  period  of 
anal^rsiB,  however,  ends  before  the  problem  was  resolved,  not  even  showing  us  to  determine 
whether  its  magnitude  was  oomparatde  to  that  of  the  others  discussed  her 

SOURCES  OF  SURPRISES  AND  REACTIONS  TO  SURPRISES:  SOME  GENERAL 
PATTERNS 

Integration  of  each  of  the  six  subsystems  dbviously  has  a  distinct  stoiy  to  tell.  When 
we  look  across  them,  however,  certain  patterns  emerge.  In  particular,  consider  the  set  of  40 
surprises  or  problems  identified  in  this  Appendix.  (A  surprise  is  a  discrete  event;  it  .s  a 
specific  manifestation  of  risk,  which  is  the  defining  characteristic  of  a  state  in  which  bad 
things — surprises — can  happen,  but  need  not  Risk  persists  without  surprises.)  They  reveal 
useful  insii^ts  into  the  sources  of  risk  or  surprise  during  MSIP  and  the  vrays  in  which  MSIP 
developers  have  reacted  to  surprise.  Table  A.1  summarizes  these  insights.  As  we  examine 
Table  A.1,  keep  in  mind  that  the  subsystems  discussed  here  are  representative  of 


TBlileA.1 

Sources  of  and  BeacUons  to  Surprises  in  Six  Integrations 


Source  of  Surprise 

Number 

Devel(q>er’8/Integratoi'B  Reaction  to  Surprise 

Tedmical  risk 

20 

Add  resources  and  time  to  overcome  problem 
Incorporate  partial  system  and  retrofit  upgrades 
Accept  lower  capability  in  subsystem 

Seek  alternative  subsystem 

Delay  schedule 

Change  development/integration  responsibilities 
Add  pressure  and  sanctions 

Develapiiient  program 

5 

Manage  difficulties  as  they  arise 

Use  analytic  aids  to  support  development 

Shortags  (rf  tost  assets 

5 

Manage  shortage 

Attempt  to  relieve  shortage 

Specification  diange 

4 

Develop  and  integrate  new  subsystem 

4 

Anticipate  and  develop  flexibility  to  react 

Use  outcome  to  update  program 
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those  associated  with  MSIP;  they  are  not  close  to  being  a  comprehensive  list  of  those  included 
and  th^  do  not  represent  a  random  sample.^*  Furthermme,  there  are  many  ways  to  identify 
surprises  in  these  integratums;  the  method  used  here  is  unavoidahly  suhieetive.  As  a  rrault, 
we  should  be  cantkms  about  making  specific  inferences  about  MSIP  as  a  whole  on  the  basis 
of  this  sample.  With  these  caveats  in  mind,  we  can  still  gamer  useful  insifldits  from  the 
of  surprise  identified  ahove. 

Technical  Risk 

The  predominant  source  of  risk  among  the  six  subsystems  was  clearly  technical  risk — 
the  risk  that  a  t^hniaiil  development  or  integration  effort  will  not  proceed  exactly  as 
expected  because  of  problems  verifying  the  design  developed  for  integration.  Also 
predominant  were  the  number  of  responses  to  surprises  associated  with  this  risk.  The 
primary  response  to  a  surprise  appears  to  be  a  decision  to  continue  work,  but  to  add 
resources  and  time  to  the  effort.  Such  a  response  need  not  affect  expected  cost  or  schedule 
for  the  full  integration  effort  MSIP  development  and  integration  planning  expected  such 
surprises,  even  if  it  could  not  predict  where  th^  would  occur,  and  provided  resources  and 
time  to  accommodate  many  of  those  assodated  with  technical  risk. 

When  a  schedule  must  slip,  integrations  often  attempt  to  maintain  part  of  the 
schedule  by  getting  a  partially  completed  int^ration  into  place  in  a  new  configuration,  even 
if  it  does  not  allow  the  full  capability  anticipated.  This  partial  integration  measure  can  then 
be  completed  with  a  retrofit  or,  less  likely,  treated  strictly  as  a  temporary  fix  and  replaced 
entirely  when  a  fully  capable  subsystem  becomes  available.  This  approach  is  consistent  with 
the  broader  MSIP  view  of  int^ration  and  retrofit  as  a  way  to  balance  integration  and 
production-inoorpmition  costs  with  risk.  It  appears  to  be  especially  attractive  when  software 
can  be  upgraded  and  retrofitted,  periups  because  software  retrofit  is  not  generally 
considered  cosffy. 

In  many  cases,  the  development  and  integration  effort  simply  accepts  lower-than- 
expected  capability  in  a  subsystem.  As  often  as  not,  sudi  acceptance  occurs  in  the 
devefopment  program  fiir  the  subsystem  itself  and  is  then  earned  into  MSIP  as  the  leas 
capable  system  is  integrated  into  the  F-IBC/D.  That  is  not  to  say  that  MSIP  plays  no  role  in 
sudi  decisioas.  TTsers  of  the  V  IfKYD  and,  hy  implifatinni  the  desi|pieiil1ntegTatnrs  in  MSTT, 

“Tbs  iuipcissi  are  not  indspsndwit  of  one  another,  eitfasr.  Eadi  surprise  aeeodated  w^th  ASPJ 
<fftfaeSBL,fcreiimipie,afhcteseviMraloftlies«fosyetMBereriewedhere.  Intheanafyeisbdow,  we 
treat  each  seeodationcfa  surprise  with  an  individaaieuhsyBtemae  a  esperatecaee  far  anafysie,  an 
^proaeh  fiiat  aOowa  as  to  dirih^iuirii  the  wape  in  which  te  managacs  of  dUfarent  Bifaayatsms  reacted 
tosurpriaas.  In  otter  coptasta,  seme  aiteraativetreataasiitBiiifot  easily  be  prefarred.  iSso  eingleton 
cases  do  not  fit  into  any  general  catagocy  and  are  not  indudsd  in  TaUe  Al. 
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have  been  actively  involved  in  dedsiona  on  final  c^Mlality.  Aa  deacribed  above,  that 
involvement  need  n<A  be  eqtedally  viaiUe  in  the  eventa. 

Aa  indicated  in  the  table,  a  variety  of  other  reaponaea  to  technical  riak  alao  occarred, 
and,  notaUy,  few  of  them  reflect  a  behef  that  developera  or  integratora  wme  failing.  The 
mayor  exoeptiona  are  reeponsea  to  technical  difficultiea  in  the  APG-68  program,  reaponaea 
that  occurred  primarily  in  the  aubayatem  development,  not  in  ita  program  for  int^pration  into 
the  F-16C/D.  Becauae  the  F-16  SPO  waa  intimately  involved  with  that  development,  the 
above  diacuaaion  gave  it  more  attention  than  the  other  individual  aubayatem  devdopmenta. 
Perhapa  if  we  looked  more  doaely  inaide  othm*  aubasratem  developmenta,  the  uae  of  aevere 
management  preaaure  and  aanctiona  on  the  ctmtraetor  would  be  more  common. 

For  each  of  the  varioua  aurpriaea,  MSIP  managera  reaponded  to  a  apedfic 
drcumatanoe  and  developed  a  aolution  that  aerved  the  needa  of  the  program  aa  a  whole.  For 
example,  when  a  alip  in  the  AMRAAM  achedule  occurred  in  tandem  with  other  eventa  that 
led  MSn*  managera  to  be  leaa  optimiatic,  they  reaponded  by  alipping  their  own  achedule.  By 
the  time  that  additional  AMRAAM  achedule  ahpe  occurred,  however,  they  had  atructured 
MSIP  aa  a  whole  ao  that  auch  alipa  had  little  effect  on  other  parta  of  MSIP  and  hence  made 
no  further  a4iuatment8  in  their  achedule. 

Such  ac^uatmenta  point  to  the  importance  of  experienced  managers.  The  range  of 
technical  aurpriaea  experienced  in  the  integration  of  theae  aix  ayatema  and  the  range  of 
reaponaea  developed  emphaaixea  the  difficulty  of  developing  detailed  contingency  plana  for 
auch  aurpriaea  in  a  development  program  like  MSIP.  It  alao  emphaaizea  the  importance  of 
having  an  experienced  ataff  working  in  a  well-atructured  aetting  to  deal  with  auch 
oontingendea  individually,  aa  they  ariae. 

DwalopfiMnt  PraQram 

The  next  two  aouroea  of  riak  originate  in  the  development  prooeaa  and  are  inherent 
reauKa  ct  the  devdtynnent  and  integratiim  annroadi  employed  in  MSIP. 

The  firat  aouroe,  'development  program,”  concerna  liaks  aaaodated  with  concurrent 
integratkmofmany  anbayatems.  It  ahowa  that  the  ooncemegtyreaaed  by  MSIP  managera, 
that  purauing  many  concurrent  integrationa  would  raiae  the  level  of  riak  for  MSIP  aa  a 
iriiole,  waareaaeBabte.  In  the  aample  preaented  here,  the  prototype  aubeyatema  uaed  to  teat 
one  another  were  rarely  equally  mature;  tyincaUy,  earlier  veraiona  of  a  aet  of  aubasratema 
weuM  be  naad  to  teat  a  new  veraion  of  a  particular  aubayatem.  Sudi  a  ineblem  ia 
unanmidable  unleaa  the  developer  waita  for  each  aet  of  improvementa  in  all  aubayotema  to  be 
complete  befeNtaating  any  of  them.  But  waiting  iaineouaiatent  with  the  teatemalyxe-fix 


(TAF)  approadi  to  development  that  emphanaea  the  need  for  oontinuoaa,  iterative  empirical 
testing  to  identify  and  resolve  tedmical  problmns.  M8IP  dmee  to  use  a  TAF  approach  and 
eqierimoed  the  risks  identified  here  as  a  result.  Using  another  approach  would  presumably 
have  generated  its  own  set  of  imUems. 

The  primary  response  to  development-program-rdhted  surprises  was  to  manage  each 
one  as  well  as  possiUe  on  its  own  terms.  MSIP  managers  repeatedly  developed  work¬ 
arounds  to  overcome  inconsistencies  amtmg  subsystems  being  tested  together.  They  looked 
for  opportunities  to  develop  valid  empiricai  results  based  on  leas  mature  test  assets  whenever 
possible  and  opportunities  to  use  less  mature  test  assets  to  make  useful  inferences  that 
would  expedite  future  emfurical  testing.  The  availability  of  analytic  tods,  especially  General 
Dynamics'  $]rstems  Integration  Laboratory  and  F-16C/D  simulator,  facilitated  the 
cocffdination  of  difforing-maturify  subsystems. 

The  general  condusion  here  is,  again,  that,  although  analytic  tools  and  contingen<y 
planning  can  hdp  greatly,  a  concurrent  integration  like  MSIP  necessarily  leaves  many 
residual  that  only  experienced  msnagers  can  handle  well.  The  development  approach 
in  MSn*  could  not  have  succeeded  without  such  managers. 

Shortage  of  Test  Assets 

The  second  category  of  develr^ment-process-related  rides,  "shortage  of  test  assets,”  is 
unavddaUe  in  any  endeavor,  particularly  when  the  partidpants  do  not  actually  pay  for  the 
use  of  the  assets  employed.  Hence,  the  existence  of  shortages  per  se  is  not  evidence  of  a 
problem.  But  when  a  scarce  resource  must  be  allocated  among  competing  uses,  decisions 
mustbemadeonacontinuingbasis.  In  MSIP,  shortages  of  many  test  assets  were  not  easy  to 
predict,  creating  surprises  that  had  to  be  managed  like  any  other  source  of  risk. 

Various  test  assets  were  important  to  MSIP,  but  particulariy  the  SIL  and  test  aircraft 
used  as  platforms  for  individual  subsystem  devdiqiments  and  the  intention  of  subeystems 
in  BISIP.  Because  they  served  many  individual  development  programs,  access  to  them  had 
to  be  allocated  among  users.  Althoui^  the  general  availabilify  of  the  two  assets  could 
typically  be  planned  with  little  difficulty,  individual  demands  on  them  were  mote 
problematic.  Donands  diangsd  as  surprises  in  individual  programs  affected  the  programs’ 
readiness  to  use  these  cmnnMm  assets  and  the  nature  of  tests  that  they  wanted  to  omduct  m 
them.  Changes  in  individual  programs  affected  aggregate  demand  on  tile  common  assets  and 
activities  required  to  customise  the  CMnmon  assets  fiar  use  in  specific  tests. 

Periuqw  IsM  obvious  as  test  assets  are  the  Bttbqystem  prototypes  develiqied  as  part  of 
MSIP  and  its  associated  subsystom  develoinients.  They  were  required  to  create  the  test 
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«nviranm«nt  for  otlMr  subqrstemB  to  be  iategrated  with  them.  Both  the  availability  such 
prototypes  and  the  pattern  of  demand  placed  on  them  were  uncertain  during  MSIP. 
Managers  continuously  updated  specific  test  sdiedules  to  accommodate  changes  on  both 
sides  of  such  a  matchup.  They  also  developed  work-arounds  to  prevent  the  test  inogram 
fifom  bogging  down  when  specific  test  assets  were  simfdy  not  availaUe. 

Leas  obvious  still  is  a  test  asset  like  the  constdlatfam  of  OPS  satdliteB,  which  was 
required  to  allow  final  testing  in  the  GPS  developmoit  and  integratim  ^wts,  but  was  well 
besnmd  the  control  of  anyone  in  BfSIP. 

In  the  subsystems  studied  here,  MSIP  managers  typically  responded  to  these  sorts  of 
shortages  by  "»an»ging  them,  allocating  access  to  cmnmon  resources  and  rescheduling  the 
iise  of  other  test  assets  to  the  <^h«nginp  drcumstances  in  MSIP.  They  also  fought  to 
expand  the  availability  of  test  assets,  particularly  test  aircraft  on  loan  from  the  Tactical  Air 
Conunand.  We  cannot  aaaess  whether  the  level  of  test  assets  actually  made  available  to 
MSIP  has  been  the  ri^t  one.  But  aignificantly  nvpnnding  thft  availalwlity  of  asseta  was 
usually  not  ^  rqition;  management  of  availaUe  assets  in  the  face  of  uncertainty  was  the  key 
risk  management  problem  here. 

Specification  CtwiQa 

In  each  case  of  the  next  source  of  risk  shown  in  Table  A.  1,  “specification  change,”  an 
external  agenqr  mandated  a  change  in  the  specifications  for  the  F-16C/D.  In  one  APG-68 
case,  the  MSIP  managers  made  a  counterprtqwsal,  which  was  ultimately  accepted.  But  in  all 
cases,  these  directives  to  change  specifications  initiated  an  orderly  process  that  created  an 
integratirm  plan,  drew  up  a  contractual  vdiide,  altered  organizational  relationships  as 
necessary,  and  implemented  the  integratirm  as  an  integral  part  of  MSIP.  The  standard 
MSIP  structure  and  procedures  were  designed  to  accommodate  such  changes,  and  they 
appear  to  have  proceeded  without  di£Bculty. 

HnWViQllv 

The  final  source  of  risk  listed  in  TaMe  A.1,  “milestone,”  is  associated  with  a  known 
date  in  the  fbture,  when  MSIP  managers  know  that  new  infimnation  will  become  availaUe. 
The  risk  lies  in  uncertainty  about  the  contents  that  inftrmation.  Our  sample  does  not 
indude  many  exautyles  of  this  “source  of  risk,”  so  perhaps  the  consistency  we  observe  in 
MSIFs  remcnae  to  milestones  is  deceptive.  In  this  samide,  BfSIP  managers  reacted  to 
nutestimas  in  two  wtys. 

yiT«t,tlMypriqiTwi  fig  tif  data  hy  making  frangmufai  that  wimMfaeilitatB 

continuing  integratkin,  no  matter  what  the  outcome  at  the  milestoae.  In  the  GPS  program. 
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MSIP  made  oontractual  arrangementa  that  would  aeomimodate  either  winner  of  the  user- 
equipment  eompetiticm.  In  the  engine  progrmn,  MSIP  development  of  an  interfiux,  the 
oonfigured  engine  bay,  that  could  accommodate  either  winnn^  of  the  engine  competition. 

Second,  perhaps  where  such  risk-reducing  arrangements  are  not  posmUe,  they  delayed 
their  own  decisions  until  a  milestone  provided  new  information.  Only  after  the  first  engine 
competition  revealed  General  Electric  as  the  sole  engine  supplier  did  MSIP  focus  on  tailoring 
the  F-16CVD  to  the  Genmral  Electric  engine.  Only  when  Pntt  and  Whitney  also  became  a 
supplier  did  the  MSIP  consider  aimilar  arrangements  for  their  engine. 

Other  Points 

Perhaps  most  obviously  miimng  fit>m  this  list  is  risk  associated  with  overall  funding 
for  MSIP.  MSIP  experienced  severe  budget  cuts  in  1987,  but  their  effects  do  not  appear  in 
the  subqrstem  histories  offered  here.  Given  concerns  in  the  SPO  that  the  budget  cut  would 
severely  affect  its  ability  to  manage  the  test  program  for  MSIP,  that  absence  is  surprising. 
Perhaps  the  effects  of  budget  cuts  would  become  more  evident  if  we  traced  the  histories  of  the 
subsystems  beyond  1988  and,  in  effect,  beyond  Block  40.  The  budget  cuts  clearly  had  a  mtgor 
effect  on  Block  70,  which  was  scaled  back  to  become  a  significantly  less  ambitious  Block  50. 
More  generally,  however,  it  is  clear  that  MSIP  proceeded  in  a  favorable  budget  environment 
until  1987.  Budget  changes  could  easily  provide  a  more  important  source  of  risk  in  similar 
analyses  of  other  developments. 

Looked  at  as  a  whole,  this  set  of  surprises  emphasizes  the  pervasive  and  continuing 
nature  of  uncertainty  during  the  MSIP.  Without  discounting  the  importance  of  MSIP’s 
institutional  preparations  to  deal  with  such  uncertainty  in  its  management  plan,  contracts, 
test  plan,  and  other  arrangements,  MSIP  could  deal  with  all  the  surprises  it  experienced  only 
because  it  had  a  weU-establiahed  and  experienced  team  of  managers.  Those  managers  were 
available  to  deal  with  individual  contingencies  as  they  arose,  flexibly  using  the  management 
structure  that  was  in  place  to  treat  each  contingency  on  its  own  terms. 

SUMMARY 

The  integrations  (ff  these  six  subsystems  illustrate  many  of  the  management  principles 
eaplained  in  Section  5.  The  surprises  above  help  document  the  practical  application  of  the 
psindides  advanced  in  MSIP  planning  documents.  For  example,  three  of  the  six  sample 
scdiaystema  are  derived  flmn  earlier  systems.  Although  their  derivative  character  did  not 
avoid  problems  during  integration,  it  helped  hmit  the  effscts  of  problems  by  providing  a 
ready  eontext  in  whidi  to  weA.  solutions  if  a  developmoit  did  not  proceed  as  expected. 
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Of  the  subqretems  preaoated  here,  the  three  that  required  the  most  complex 
int^ration  with  other  systems— the  APG-68,  AMRAAM,  and  LANTIRN— all  present  similar 
problons  above:  All  suffered  from  the  problem  that  MSIP  planners  had  eqiected  would 
introduce  serious  risks  into  the  devdopment  of  the  F-16C/D — the  concurrence  of  many 
interrelated  develofmient  and  integration  pn^rams,  which  was  manifested  in  shortages  of 
test  assets  for  integrating  several  subss^stems  at  mice  and  the  difficulty  of  obtaining  mature 
enough  sub^stem  protot3rpe8  to  test  the  latest  upgrades  of  other  subqrartem  prototypes. 

These  subsystem  integrations  illustrate  how  heavily  MSIP  relied  on  incorporating 
subsystems  with  partial  capabilities  to  maintain  its  schedule.  In  feet,  MSIP  had  to  be 
coordinated  with  a  set  of  production  contracts  that  required  a  steady  rate  of  F-16  production. 
When  MSIPs  integration  efforts  failed  to  reach  their  expectations,  the  program  stiU  had  to 
field  a  producible  configuration  that  could  fly  safely.  Incorporating  subsystems  with  partial 
capabilities  effectively  put  placeholders  in  the  fleet  that  could  be  retrofitted  or  replaced  in  the 
future,  when  MSIP  activities  made  full  capabilities  available. 

All  these  subs]rstem  integrations  testify  to  the  variety  of  surprises  that  arose  in  the 
program.  MSIP  expected  such  variety  and  prepared  for  it  in  two  ways:  It  established  a 
flexible  management  structure  to  accept  surprises  as  they  came  and  work  with  them,  and  it 
maintained  an  experienced  management  staff  to  improvise  solutions  to  the  problems 
presented  by  those  surprises.  The  success  of  this  approach  is  evident  in  the  above  short 
histories.  Despite  substantial  turbulence  in  the  developments  of  many  of  these  systems  and 
continuing  surprises  during  their  integrations,  MSIP  ultimately  completed  the  integrations 
successfully  and  achieved  its  principal  goals  for  the  F-16C/D  development. 
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